01:48:09,422 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:48:09,432 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3275940955677406053] 01:48:09,432 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:48:09,433 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:48:09,433 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:48:09,435 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:09,435 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:09,448 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:48:09,448 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:48:09,448 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:48:09,469 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:09,483 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-7637547413978126307] 01:48:09,483 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:48:09,490 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:09,490 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:48:09,490 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:09,490 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:09,534 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:09,553 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-7267816473472568440] 01:48:09,553 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:48:09,560 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:48:09,568 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:09,568 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:48:09,569 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:09,569 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:09,571 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:09,571 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:09,575 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:09,576 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithMultipleShards 01:48:09,600 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-608258690] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:09,600 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithMultipleShards 01:48:09,604 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: no initial generation override, starting from 0 01:48:09,604 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:48:09,605 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5d0bc23 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@167b98d9 01:48:09,616 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithMultipleShards 01:48:09,616 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-1-shard-cars-testTransactionChainWithMultipleShards 01:48:09,616 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-1-shard-people-testTransactionChainWithMultipleShards 01:48:09,617 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:09,617 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:48:09,617 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:48:09,618 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@56ca37e8 01:48:09,618 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionChainWithMultipleShards, 0, 1710164889605, None)) 01:48:09,618 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164889604,0,0)) 01:48:09,618 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@167b98d9 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@551fb3da 01:48:09,619 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} started 01:48:09,619 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:09,621 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:09,623 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithMultipleShards 01:48:09,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@17fe0c3 01:48:09,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:09,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionChainWithMultipleShards 01:48:09,624 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:48:09,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:48:09,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-cars-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards on shard member-1-shard-cars-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-189726805] 01:48:09,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testTransactionChainWithMultipleShards 01:48:09,624 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:48:09,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:48:09,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-people-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards on shard member-1-shard-people-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-675386581] 01:48:09,625 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-people-testTransactionChainWithMultipleShards, persistent : true 01:48:09,625 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-people-testTransactionChainWithMultipleShards with journal batch size 1 01:48:09,633 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:48:09,633 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-675386581] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:09,633 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-notifier#-1933048042] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:09,635 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:09,635 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:09,637 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:09,647 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#876458727] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:09,648 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithMultipleShards 01:48:09,649 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [people] 01:48:09,649 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-1-shard-people-testTransactionChainWithMultipleShards is initialized 01:48:09,649 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionChainWithMultipleShards, persistent : true 01:48:09,649 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-189726805] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:09,649 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionChainWithMultipleShards with journal batch size 1 01:48:09,649 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-notifier#-483725764] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:09,649 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#1449033971] 01:48:09,650 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#1449033971] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:09,651 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:09,651 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:09,651 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:48:09,651 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 01:48:09,651 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [cars] 01:48:09,651 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionChainWithMultipleShards is initialized 01:48:09,651 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-notifier#-1933048042 created and ready for shard:member-1-shard-people-testTransactionChainWithMultipleShards 01:48:09,652 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#-1779926199] 01:48:09,652 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#-1779926199] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:09,671 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:48:09,671 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionChainWithMultipleShards is no longer the leader 01:48:09,671 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from null to Follower 01:48:09,671 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 01:48:09,676 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: no initial generation override, starting from 0 01:48:09,676 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:48:09,676 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@302dfa9f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@a68900c 01:48:09,676 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@71947b19 01:48:09,677 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithMultipleShards 01:48:09,677 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-2-shard-cars-testTransactionChainWithMultipleShards 01:48:09,677 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-2-shard-people-testTransactionChainWithMultipleShards 01:48:09,677 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:09,677 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:48:09,677 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:48:09,677 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionChainWithMultipleShards, 0, 1710164889677, None)) 01:48:09,677 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164889676,0,0)) 01:48:09,678 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@a68900c to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4dcba880 01:48:09,678 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} started 01:48:09,678 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@55a4d517 01:48:09,678 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:09,678 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionChainWithMultipleShards 01:48:09,679 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from null to Follower 01:48:09,684 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-notifier#-483725764 created and ready for shard:member-1-shard-cars-testTransactionChainWithMultipleShards 01:48:09,684 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from null to Follower 01:48:09,684 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 01:48:09,684 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from null to Follower 01:48:09,688 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:48:09,688 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:48:09,688 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-cars-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards on shard member-2-shard-cars-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards#1417157206] 01:48:09,688 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testTransactionChainWithMultipleShards 01:48:09,688 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:48:09,688 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:48:09,688 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-people-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards on shard member-2-shard-people-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards#-515179218] 01:48:09,688 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionChainWithMultipleShards, persistent : true 01:48:09,689 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards#1417157206] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:09,689 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-notifier#1235346320 created and ready for shard:member-2-shard-cars-testTransactionChainWithMultipleShards 01:48:09,689 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionChainWithMultipleShards with journal batch size 1 01:48:09,689 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-people-testTransactionChainWithMultipleShards, persistent : true 01:48:09,689 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-notifier#1235346320] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:09,689 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-notifier#-73726526 created and ready for shard:member-2-shard-people-testTransactionChainWithMultipleShards 01:48:09,691 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards#-515179218] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:09,691 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-notifier#-73726526] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:09,691 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-1074441617] 01:48:09,691 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-1074441617] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:09,694 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:09,695 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:09,695 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:48:09,695 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 01:48:09,695 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithMultipleShards , received role change from null to Follower 01:48:09,695 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:48:09,695 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [cars] 01:48:09,695 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionChainWithMultipleShards is initialized 01:48:09,695 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 01:48:09,696 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-people-testTransactionChainWithMultipleShards with journal batch size 1 01:48:09,696 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-1600270462] 01:48:09,696 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-1600270462] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:09,697 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-2-shard-cars-testTransactionChainWithMultipleShards from null to Follower 01:48:09,698 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:09,698 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:09,698 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:48:09,698 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionChainWithMultipleShards is no longer the leader 01:48:09,698 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-1-shard-people-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:48:09,698 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [people] 01:48:09,698 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-people-testTransactionChainWithMultipleShards is initialized 01:48:09,698 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionChainWithMultipleShards , received role change from null to Follower 01:48:09,698 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 01:48:09,698 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-2-shard-people-testTransactionChainWithMultipleShards from null to Follower 01:48:09,728 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-608258690] 01:48:09,728 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: findLocalShard : cars 01:48:09,728 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-189726805] 01:48:09,904 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:48:09,904 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:48:09,904 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionChainWithMultipleShards] 01:48:09,904 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 01:48:09,904 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 1 01:48:09,904 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:09,904 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Closing 01:48:09,904 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:48:09,904 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 01:48:09,905 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from Follower to Candidate 01:48:09,905 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from Follower to Candidate 01:48:09,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:48:09,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:48:09,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testTransactionChainWithMultipleShards] 01:48:09,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 01:48:09,943 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 1 01:48:09,943 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:09,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Closing 01:48:09,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:48:09,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionChainWithMultipleShards is no longer the leader 01:48:09,943 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from Follower to Candidate 01:48:09,943 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from Follower to Candidate 01:48:10,143 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:48:10,143 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 01:48:10,143 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 2 01:48:10,217 PM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:48:10,243 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:48:10,243 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 01:48:10,243 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 2 01:48:10,247 PM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:48:10,433 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:48:10,433 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 01:48:10,433 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 3 01:48:10,456 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:48:10,456 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:48:10,457 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:10,457 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards 01:48:10,457 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:48:10,457 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards 01:48:10,457 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:48:10,457 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:10,457 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:48:10,457 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:48:10,457 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:48:10,457 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:48:10,457 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:48:10,457 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-people-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:48:10,469 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:10,469 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards 01:48:10,469 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:48:10,469 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards 01:48:10,469 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:48:10,469 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:10,469 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:48:10,469 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:48:10,469 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:48:10,469 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:48:10,533 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:48:10,533 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 01:48:10,533 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 3 01:48:10,533 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testTransactionChainWithMultipleShards 01:48:10,538 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:10,539 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Cluster state: [] 01:48:10,539 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:48:10,539 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:48:10,539 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=null 01:48:10,539 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 01:48:10,539 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 01:48:10,539 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 01:48:10,543 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 01:48:10,543 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 01:48:10,543 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testTransactionChainWithMultipleShards] 01:48:10,543 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:10,543 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 01:48:10,544 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:48:10,544 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: became leader with frontend state for [] 01:48:10,544 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from Candidate to Leader 01:48:10,544 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 01:48:10,544 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:10,544 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:10,544 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from Candidate to Leader 01:48:10,547 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionChainWithMultipleShards: Closing 01:48:10,547 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testTransactionChainWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:48:10,548 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-people-testTransactionChainWithMultipleShards status sync done false 01:48:10,548 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 01:48:10,548 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:10,548 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:10,551 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:48:10,551 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:10,584 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:48:10,667 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-people-testTransactionChainWithMultipleShards status sync done true 01:48:10,703 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:48:10,703 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 01:48:10,703 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 4 01:48:10,703 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionChainWithMultipleShards 01:48:10,705 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:10,706 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Cluster state: [] 01:48:10,706 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:48:10,706 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:48:10,706 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=null 01:48:10,706 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:48:10,706 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 01:48:10,706 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:48:10,711 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:48:10,711 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:48:10,711 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionChainWithMultipleShards] 01:48:10,711 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:10,711 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:48:10,711 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:48:10,711 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: became leader with frontend state for [] 01:48:10,713 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithMultipleShards: Closing 01:48:10,713 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testTransactionChainWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:48:10,713 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-cars-testTransactionChainWithMultipleShards status sync done false 01:48:10,713 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 01:48:10,713 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:10,713 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:10,713 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All Shards are ready - data store testTransactionChainWithMultipleShards is ready 01:48:10,717 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:48:10,717 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:10,718 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from Candidate to Leader 01:48:10,718 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 01:48:10,718 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:10,718 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:10,718 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from Candidate to Leader 01:48:10,718 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All Shards are ready - data store testTransactionChainWithMultipleShards is ready 01:48:10,785 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-608258690] 01:48:10,786 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: findLocalShard : people 01:48:10,786 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-675386581] 01:48:10,787 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionChainWithMultipleShards commit rateLimit adjusted to 176.34356011097387 01:48:10,787 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,787 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#876458727] 01:48:10,787 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:48:10,787 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:48:10,787 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#876458727] 01:48:10,787 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards, primaryVersion=13] 01:48:10,787 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:48:10,787 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:48:10,787 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:48:10,787 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards, primaryVersion=13] 01:48:10,787 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards, primaryVersion=13] 01:48:10,788 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:48:10,788 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Readying 2 components for commit 01:48:10,788 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 01:48:10,788 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:48:10,788 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Readying transaction for shard people 01:48:10,788 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:48:10,788 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] for shard cars 01:48:10,788 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 canCommit 01:48:10,788 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] found - creating WRITE_ONLY transaction context 01:48:10,788 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:48:10,788 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:48:10,788 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 = [ 2 ] 01:48:10,788 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction called 01:48:10,788 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 sending 2 batched modifications, ready: true 01:48:10,788 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] 01:48:10,788 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards, primaryVersion=13] 01:48:10,788 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] for shard people 01:48:10,788 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] found - creating WRITE_ONLY transaction context 01:48:10,788 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 01:48:10,788 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person 01:48:10,788 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 = [ 2 ] 01:48:10,789 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction called 01:48:10,789 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 sending 2 batched modifications, ready: true 01:48:10,789 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] 01:48:10,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, disabling transaction tracking 01:48:10,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 01:48:10,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 01:48:10,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:48:10,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:10,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 01:48:10,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}]]] 01:48:10,794 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 01:48:10,794 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 01:48:10,794 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:48:10,794 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 01:48:10,794 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@628464fb 01:48:10,794 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying 2 batched modifications for Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,794 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Readying Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 of 2 operations, client version 13 01:48:10,794 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,794 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 ready 01:48:10,794 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 01:48:10,794 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, disabling transaction tracking 01:48:10,794 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 01:48:10,794 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}]]] 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@7d3f2b71 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying 2 batched modifications for Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Readying Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 of 2 operations, client version 13 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 ready 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE, modificationType=WRITE}}}]} 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}]]] 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 1 01:48:10,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:10,796 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 01:48:10,796 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-cars-testTransactionChainWithMultipleShards status sync done true 01:48:10,797 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction 01:48:10,797 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-189726805)] resolved 01:48:10,797 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:48:10,797 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}]]] 01:48:10,798 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 01:48:10,798 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:48:10,798 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 01:48:10,798 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 01:48:10,798 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 1 01:48:10,798 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:10,798 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction 01:48:10,798 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-675386581)] resolved 01:48:10,798 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: successfully resolved all cohort actors 01:48:10,798 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 finishCanCommit 01:48:10,798 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:48:10,798 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-189726805)] 01:48:10,798 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-189726805)] 01:48:10,798 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:10,798 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 01:48:10,798 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}]]] 01:48:10,799 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:48:10,799 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 01:48:10,800 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:48:10,800 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:10,800 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 01:48:10,800 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}]]] 01:48:10,800 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}]]] 01:48:10,800 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 01:48:10,800 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:48:10,800 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 01:48:10,800 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 01:48:10,800 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 2 01:48:10,800 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 0 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionChainWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:48:10,801 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, nanosAgo=7825888} is already disabled 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 0 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionChainWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, nanosAgo=6764330} is already disabled 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Can committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 validated 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:10,801 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: success 01:48:10,802 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}]]] 01:48:10,802 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 01:48:10,802 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:48:10,802 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 01:48:10,802 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 01:48:10,802 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 2 01:48:10,802 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:10,802 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 01:48:10,803 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:48:10,803 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-675386581)] 01:48:10,803 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-675386581)] 01:48:10,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:48:10,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:48:10,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 1 01:48:10,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 01:48:10,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 01:48:10,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} has no callback 01:48:10,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 3 01:48:10,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:10,805 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:48:10,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:48:10,805 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:48:10,805 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 1 01:48:10,805 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 01:48:10,805 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 01:48:10,805 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} has no callback 01:48:10,805 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:48:10,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Can committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,805 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:10,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 validated 01:48:10,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:10,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: success 01:48:10,805 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 1 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 0 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionChainWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 1 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 3 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 1 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 0 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionChainWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 1 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:48:10,807 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:48:10,808 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:48:10,808 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit returning result: true 01:48:10,808 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 commit 01:48:10,808 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 finish commit 01:48:10,808 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-189726805)] 01:48:10,808 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-189726805)] 01:48:10,808 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-675386581)] 01:48:10,808 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-675386581)] 01:48:10,813 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,813 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,813 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,813 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,813 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 candidate ready 01:48:10,813 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit - candidate rootPath: / 01:48:10,813 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 candidate ready 01:48:10,814 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit - candidate rootPath: / 01:48:10,814 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit - messages: [] 01:48:10,814 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: preCommit - successfulFromPrevious: [] 01:48:10,814 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 prepared 01:48:10,814 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:10,814 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,814 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit - messages: [] 01:48:10,814 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,814 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: preCommit - successfulFromPrevious: [] 01:48:10,814 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 prepared 01:48:10,814 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:10,814 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,814 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,814 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 180 01:48:10,814 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}] 01:48:10,814 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}] 01:48:10,814 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 176 01:48:10,814 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:10,814 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=176}] 01:48:10,814 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=176}] 01:48:10,814 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:48:10,814 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 01:48:10,814 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:10,814 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}]]] 01:48:10,814 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 submitted to persistence 01:48:10,814 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:48:10,814 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 01:48:10,814 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=176}]]] 01:48:10,814 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 submitted to persistence 01:48:10,816 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}] 01:48:10,816 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=176}] 01:48:10,816 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}]]] 01:48:10,816 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 01:48:10,816 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:48:10,816 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=180} 01:48:10,816 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}] 01:48:10,817 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 2 01:48:10,817 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=2, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:10,817 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=176}]]] 01:48:10,817 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 01:48:10,817 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:48:10,817 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=176} 01:48:10,817 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=176}] 01:48:10,817 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 2 01:48:10,817 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:10,817 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}] 01:48:10,817 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=176}] 01:48:10,818 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 2, nextIndex: 3 01:48:10,818 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:48:10,818 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 2 01:48:10,818 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 01:48:10,818 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 2 data Simple{size=180} 01:48:10,818 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 2, nextIndex: 3 01:48:10,818 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,818 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:48:10,818 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 2 01:48:10,818 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 01:48:10,819 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: commit - successfulFromPrevious: [] 01:48:10,819 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 2 data Simple{size=176} 01:48:10,819 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:48:10,819 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,819 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: commit - successfulFromPrevious: [] 01:48:10,819 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:48:10,819 PM [cluster-test-shard-dispatcher-21] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE, modificationType=WRITE}}}]} 01:48:10,819 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-people-testTransactionChainWithMultipleShards%23-675386581$f] 01:48:10,819 PM [cluster-test-shard-dispatcher-28] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 01:48:10,819 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-cars-testTransactionChainWithMultipleShards%23-189726805$e] 01:48:10,819 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#1149689] with name member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher 01:48:10,819 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-1653282326] with name member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher 01:48:10,819 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-1653282326] 01:48:10,819 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 3 01:48:10,819 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 4 01:48:10,819 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:48:10,819 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:48:10,819 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#1149689] 01:48:10,819 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:48:10,819 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 01:48:10,819 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 01:48:10,819 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:48:10,820 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 168.7 μs 01:48:10,820 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-1653282326] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:10,820 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 92.52 μs 01:48:10,820 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#1149689] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:10,821 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: commit succeeded 01:48:10,821 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 2 01:48:10,821 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,821 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:48:10,821 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 2 01:48:10,821 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 01:48:10,821 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 2 data Simple{size=176} 01:48:10,822 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,822 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:10,822 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:48:10,822 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] found - trying create transaction 01:48:10,822 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] 01:48:10,822 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 2 01:48:10,822 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:48:10,822 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 2 01:48:10,822 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 01:48:10,822 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Queuing TransactionOperation 01:48:10,822 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 2 data Simple{size=180} 01:48:10,822 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:10,822 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] for shard people, trying to use DirectTransactionContextWrapper 01:48:10,822 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:48:10,822 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] found - trying create transaction 01:48:10,822 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] 01:48:10,822 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#184434530] with name member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher 01:48:10,822 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Queuing TransactionOperation 01:48:10,822 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Queuing TransactionOperation 01:48:10,822 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 4 01:48:10,822 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 01:48:10,822 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#989851322] with name member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher 01:48:10,822 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 3 01:48:10,822 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 01:48:10,823 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#184434530] 01:48:10,823 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:48:10,823 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 65.12 μs 01:48:10,823 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#184434530] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:10,823 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#989851322] 01:48:10,823 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:48:10,823 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 44.90 μs 01:48:10,823 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#989851322] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:10,826 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Creating transaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,826 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 01:48:10,826 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@afefe51 01:48:10,826 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Creating transaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,826 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 01:48:10,826 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@3e0b9bf9 01:48:10,828 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#2082905242, transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] 01:48:10,828 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 01:48:10,828 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 01:48:10,828 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 sending 1 batched modifications, ready: false 01:48:10,828 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#2082905242)] 01:48:10,828 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#2082905242)] 01:48:10,828 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#-88936653, transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] 01:48:10,829 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] 01:48:10,830 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#2082905242] 01:48:10,830 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#2082905242] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:10,830 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#-88936653] 01:48:10,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 modifications completed with BatchedModificationsReply [numBatched=1] 01:48:10,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 ReadData operation succeeded 01:48:10,832 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] 01:48:10,832 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 sending 1 batched modifications, ready: false 01:48:10,832 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#-88936653)] 01:48:10,832 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#-88936653)] 01:48:10,835 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 modifications completed with BatchedModificationsReply [numBatched=1] 01:48:10,835 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 ReadData operation succeeded 01:48:10,835 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Readying 2 components for commit 01:48:10,835 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Readying transaction for shard cars 01:48:10,835 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 = [ 1 ] 01:48:10,835 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 readyTransaction called 01:48:10,835 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 sending 0 batched modifications, ready: true 01:48:10,835 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#2082905242)] 01:48:10,835 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Readying transaction for shard people 01:48:10,835 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 = [ 1 ] 01:48:10,835 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 readyTransaction called 01:48:10,836 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 sending 0 batched modifications, ready: true 01:48:10,836 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#-88936653)] 01:48:10,836 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:48:10,836 PM [main] [DEBUG] TransactionChainProxy - Tx: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 - waiting for ready futures with pending Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:48:10,836 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Queuing TransactionOperation 01:48:10,836 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Readying 1 components for commit 01:48:10,836 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Readying transaction for shard people 01:48:10,836 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Queuing TransactionOperation 01:48:10,836 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 canCommit 01:48:10,837 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,837 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: handleForwardedReadyTransaction for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,837 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, client version 13 01:48:10,837 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,837 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 ready 01:48:10,837 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:48:10,838 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,838 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: handleForwardedReadyTransaction for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,838 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, client version 13 01:48:10,838 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,838 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 ready 01:48:10,838 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}]} 01:48:10,840 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 readyTransaction 01:48:10,840 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-189726805)] resolved 01:48:10,840 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 modifications completed with ReadyTransactionReply [version=13] 01:48:10,840 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 readyTransaction 01:48:10,840 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-675386581)] resolved 01:48:10,840 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: successfully resolved all cohort actors 01:48:10,840 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 finishCanCommit 01:48:10,840 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-189726805)] 01:48:10,840 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-189726805)] 01:48:10,840 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransactionChainProxy - Tx: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 - previous Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 readied - proceeding to FindPrimaryShard 01:48:10,840 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] for shard people 01:48:10,840 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] found - creating WRITE_ONLY transaction context 01:48:10,840 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 executeDelete called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] 01:48:10,840 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 directCommit called 01:48:10,840 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 sending 1 batched modifications, ready: true 01:48:10,840 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] 01:48:10,841 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 modifications completed with ReadyTransactionReply [version=13] 01:48:10,842 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Can committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,842 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,842 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 validated 01:48:10,842 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:10,842 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: success 01:48:10,843 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Reusing a chained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 01:48:10,843 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 snapshot MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}] 01:48:10,843 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:48:10,843 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Readying Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 of 1 operations, client version 13 01:48:10,843 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:48:10,843 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 ready 01:48:10,843 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=DELETE}}}}}}}]} 01:48:10,843 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 is scheduled for canCommit step 01:48:10,848 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:48:10,848 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-675386581)] 01:48:10,848 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-675386581)] 01:48:10,849 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Can committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,849 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,849 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 validated 01:48:10,849 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:10,849 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: success 01:48:10,850 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:48:10,850 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit returning result: true 01:48:10,852 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 commit 01:48:10,852 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 finish commit 01:48:10,852 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-189726805)] 01:48:10,852 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-189726805)] 01:48:10,852 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-675386581)] 01:48:10,852 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-675386581)] 01:48:10,854 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,854 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,854 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 candidate ready 01:48:10,854 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit - candidate rootPath: / 01:48:10,854 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit - messages: [] 01:48:10,854 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: preCommit - successfulFromPrevious: [] 01:48:10,854 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 prepared 01:48:10,854 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:10,854 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,855 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,855 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 235 01:48:10,855 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=235}] 01:48:10,855 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=235}] 01:48:10,855 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:10,855 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 01:48:10,855 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 01:48:10,855 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=235}]]] 01:48:10,855 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 submitted to persistence 01:48:10,856 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,856 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,856 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 candidate ready 01:48:10,856 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit - candidate rootPath: / 01:48:10,856 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit - messages: [] 01:48:10,856 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: preCommit - successfulFromPrevious: [] 01:48:10,856 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 prepared 01:48:10,856 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:10,856 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,856 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,856 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=235}] 01:48:10,856 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 224 01:48:10,856 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:48:10,856 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 validated 01:48:10,856 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:10,856 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: success 01:48:10,856 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:48:10,856 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:48:10,857 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=235}]]] 01:48:10,857 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 01:48:10,857 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 candidate ready 01:48:10,857 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:48:10,857 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: canCommit - candidate rootPath: / 01:48:10,857 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=235} 01:48:10,857 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=235}] 01:48:10,857 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 2 01:48:10,857 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: canCommit - messages: [] 01:48:10,857 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:10,857 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: preCommit - successfulFromPrevious: [] 01:48:10,857 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 prepared 01:48:10,857 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:10,857 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:48:10,857 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 scheduled for commit step 01:48:10,857 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}] 01:48:10,857 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}] 01:48:10,857 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: false 01:48:10,857 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 submitted to persistence 01:48:10,857 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:48:10,857 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=235}] 01:48:10,857 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 221 01:48:10,857 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}] 01:48:10,857 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}] 01:48:10,858 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:10,858 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 01:48:10,858 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 3 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 01:48:10,858 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=3, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}], SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}]]] 01:48:10,858 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 submitted to persistence 01:48:10,859 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 3, nextIndex: 4 01:48:10,859 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 01:48:10,859 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 3 01:48:10,859 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 01:48:10,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=235} 01:48:10,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: commit - successfulFromPrevious: [] 01:48:10,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 01:48:10,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:48:10,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-cars-testTransactionChainWithMultipleShards%23-189726805$r] 01:48:10,860 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:48:10,860 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=3, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}], SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}]]] 01:48:10,860 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 01:48:10,860 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:48:10,860 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=224} 01:48:10,860 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}] 01:48:10,860 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=221} 01:48:10,860 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}] 01:48:10,860 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 119.1 μs 01:48:10,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 2 for term 4 01:48:10,860 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 3 01:48:10,860 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=4, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:10,860 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}] 01:48:10,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:48:10,860 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}] 01:48:10,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=3 01:48:10,861 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}] 01:48:10,861 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}] 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 4, nextIndex: 5 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 2 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 3 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=224} 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: commit - successfulFromPrevious: [] 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}]} 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-people-testTransactionChainWithMultipleShards%23-675386581$s] 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 4 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 4 data Simple{size=221} 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: commit - successfulFromPrevious: [] 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, modificationType=DISAPPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=DELETE, modificationType=DELETE}}}}}}}]} 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-people-testTransactionChainWithMultipleShards$p] 01:48:10,863 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 3 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 3 01:48:10,863 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 01:48:10,863 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 3 01:48:10,863 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:48:10,863 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=235} 01:48:10,863 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 01:48:10,864 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,864 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:48:10,864 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 75.79 μs 01:48:10,864 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:48:10,864 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 36.16 μs 01:48:10,864 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 2 for term 4 01:48:10,864 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=3 01:48:10,865 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:48:10,865 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 88.47 μs 01:48:10,867 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: commit succeeded 01:48:10,868 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 modifications completed with CommitTransactionReply [version=13] 01:48:10,868 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 canCommit 01:48:10,868 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 successfully completed direct commit 01:48:10,868 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 01:48:10,868 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:10,868 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:48:10,868 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] found - trying create transaction 01:48:10,868 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] 01:48:10,868 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 01:48:10,868 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 4 01:48:10,868 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 01:48:10,868 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 3 01:48:10,868 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 01:48:10,869 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=224} 01:48:10,869 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:48:10,869 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 4 01:48:10,869 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 01:48:10,869 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 4 data Simple{size=221} 01:48:10,869 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:48:10,869 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 3 01:48:10,870 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 01:48:10,870 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:48:10,870 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 72.53 μs 01:48:10,870 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:48:10,870 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 34.23 μs 01:48:10,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 01:48:10,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Creating transaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 01:48:10,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 01:48:10,873 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:0_6#-1528998637] 01:48:10,874 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:0_6#-1528998637, transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0, version=13] 01:48:10,874 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:48:10,874 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:0_6#-1528998637)] 01:48:10,876 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 ReadData operation succeeded 01:48:10,876 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] for shard people, trying to use DirectTransactionContextWrapper 01:48:10,876 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:48:10,876 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] found - trying create transaction 01:48:10,876 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] 01:48:10,876 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 01:48:10,877 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Creating transaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 01:48:10,877 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:0_7#2094886010] 01:48:10,878 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:0_7#2094886010, transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0, version=13] 01:48:10,878 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] 01:48:10,878 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:0_7#2094886010)] 01:48:10,880 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 ReadData operation succeeded 01:48:10,880 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:48:10,881 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testTransactionChainWithMultipleShards 01:48:10,881 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testTransactionChainWithMultipleShards 01:48:10,881 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithMultipleShards - waiting on 2 shards 01:48:10,881 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onShutDown 01:48:10,881 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onShutDown 01:48:10,881 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Initiating leader transfer 01:48:10,881 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Initiating leader transfer 01:48:10,883 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:48:10,883 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:10,883 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:10,883 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:48:10,883 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:10,883 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:10,884 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2a5e0067 01:48:10,884 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2a5e0067 01:48:10,884 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Attempting to transfer leadership 01:48:10,885 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5efaead4 01:48:10,885 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5efaead4 01:48:10,885 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Attempting to transfer leadership 01:48:10,885 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:48:10,885 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:48:10,887 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards] 01:48:10,887 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards] 01:48:10,887 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:48:10,887 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:10,887 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:10,887 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:48:10,887 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:10,887 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:10,888 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionChainWithMultipleShards, matchIndex: 3, lastIndex: 3, isVoting: true 01:48:10,888 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:48:10,888 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:48:10,888 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Leader transfer complete 01:48:10,889 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testTransactionChainWithMultipleShards, matchIndex: 4, lastIndex: 4, isVoting: true 01:48:10,889 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:48:10,889 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:48:10,889 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Leader transfer complete 01:48:11,293 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All shards shutdown - sending PoisonPill to self 01:48:11,293 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] ShardManager - shard-manager-testTransactionChainWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-189726805]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:48:11,293 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionChainWithMultipleShards 01:48:11,293 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader)_slicer-id-415: Closing 01:48:11,293 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 01:48:11,294 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-people-testTransactionChainWithMultipleShards 01:48:11,294 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards (Leader)_slicer-id-414: Closing 01:48:11,294 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 01:48:11,294 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithMultipleShards 01:48:11,294 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 01:48:11,294 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:48:11,295 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 01:48:11,295 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: shutting down 01:48:11,295 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 01:48:11,296 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:11,296 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:11,310 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:11,310 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:11,311 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:11,311 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:11,312 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:48:11,337 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithMultipleShards: Closing 01:48:11,337 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionChainWithMultipleShards 01:48:11,337 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Closing 01:48:11,338 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 01:48:11,338 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-people-testTransactionChainWithMultipleShards 01:48:11,338 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Closing 01:48:11,338 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 01:48:11,338 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithMultipleShards 01:48:11,338 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:11,338 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:12,315 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-3275940955677406053] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:12,353 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-7637547413978126307] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:12,360 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:12,361 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:12,368 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:12.370] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:12,378 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:12,378 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:12,382 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:12.384] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:12,401 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:12,408 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [6175083169199875299] 01:48:12,408 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:48:12,410 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:48:12,410 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:48:12,420 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:12,420 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:12,420 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:48:12,420 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:48:12,421 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:48:12,434 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:12,445 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [2325052457705856564] 01:48:12,445 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:48:12,447 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:12,447 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:48:12,448 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:12,448 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:12,470 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:12,485 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [4434548516439982140] 01:48:12,485 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:48:12,487 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:12,487 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:48:12,495 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:12,495 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:12,497 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:12,498 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:12,500 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testInstallSnapshot 01:48:12,514 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#977628211] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:12,537 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: no initial generation override, starting from 0 01:48:12,537 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 01:48:12,538 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@30291231 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@25617cf9 01:48:12,538 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@33b1b785 01:48:12,538 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testInstallSnapshot, 0, 1710164892538, None)) 01:48:12,538 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164892537,0,0)) 01:48:12,538 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@25617cf9 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3034b017 01:48:12,539 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} started 01:48:12,539 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:12,541 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:12,542 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testInstallSnapshot 01:48:12,549 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testInstallSnapshot 01:48:12,549 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testInstallSnapshot 01:48:12,550 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Creating local shard: member-1-shard-cars-testInstallSnapshot 01:48:12,550 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:12,550 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:48:12,550 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1e6261a4 01:48:12,550 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:12,550 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testInstallSnapshot 01:48:12,550 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:48:12,550 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:48:12,550 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: updated peer member-1-shard-cars-testInstallSnapshot on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot on shard member-1-shard-cars-testInstallSnapshot whose actor address is Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#869779443] 01:48:12,551 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Shard created : member-1-shard-cars-testInstallSnapshot, persistent : true 01:48:12,551 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#869779443] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:12,551 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Starting recovery for member-1-shard-cars-testInstallSnapshot with journal batch size 1 01:48:12,553 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/shard-member-1:snapshot-read#-276502646] 01:48:12,553 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-notifier#1227356885 created and ready for shard:member-1-shard-cars-testInstallSnapshot 01:48:12,553 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/shard-member-1:snapshot-read#-276502646] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:12,558 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.SnapshotOffer from Actor[akka://cluster-test/system/in-memory-snapshot-store#-2012014745] 01:48:12,559 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: SnapshotOffer called. 01:48:12,559 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Moving last applied index from -1 to 5 01:48:12,559 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=1, votedFor=null 01:48:12,559 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Applying recovered snapshot 01:48:12,559 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendMetadata - member-1-shard-cars-testInstallSnapshot: clearing clients {} 01:48:12,560 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testInstallSnapshot: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#1326107190] with name member-1-shard-cars-testInstallSnapshot-DTCL-publisher 01:48:12,560 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#1326107190] 01:48:12,560 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testInstallSnapshot: state snapshot applied in 888.9 μs 01:48:12,560 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testInstallSnapshot: publishChanges: rootPath: / 01:48:12,560 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Recovery snapshot applied for member-1-shard-cars-testInstallSnapshot in 1.191 ms: snapshotIndex=5, snapshotTerm=1, journal-size=0 01:48:12,560 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testInstallSnapshot: Elapsed time for generation of change events for member-1-shard-cars-testInstallSnapshot-DTCL-publisher: 121.5 μs 01:48:12,561 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:12,561 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Recovery completed in 2.173 ms - Switching actor to Follower - last log index = 5, last log term = 1, snapshot index = 5, snapshot term = 1, journal size = 0 01:48:12,561 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 01:48:12,561 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testInstallSnapshot is no longer the leader 01:48:12,561 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Initializing shard [cars] 01:48:12,561 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-testInstallSnapshot is initialized 01:48:12,567 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from null to Follower 01:48:12,567 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , registered listener akka://cluster-test/user/shardmanager-testInstallSnapshot 01:48:12,569 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: no initial generation override, starting from 0 01:48:12,569 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} 01:48:12,569 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5450a9e5 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7f049e0a 01:48:12,570 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@36bf4637 01:48:12,570 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testInstallSnapshot, 0, 1710164892570, None)) 01:48:12,570 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164892569,0,0)) 01:48:12,570 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7f049e0a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2f04965c 01:48:12,571 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} started 01:48:12,576 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#2145933467] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:12,577 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testInstallSnapshot 01:48:12,578 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testInstallSnapshot 01:48:12,578 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Creating local shard: member-2-shard-cars-testInstallSnapshot 01:48:12,578 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@d9fd83c 01:48:12,578 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:12,578 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testInstallSnapshot 01:48:12,579 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testInstallSnapshot, persistent : true 01:48:12,579 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot#-749817139] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:12,579 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/shard-member-2:snapshot-read#1894954948] 01:48:12,579 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-notifier#-603218511 created and ready for shard:member-2-shard-cars-testInstallSnapshot 01:48:12,579 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-notifier#-603218511] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:12,579 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/shard-member-2:snapshot-read#1894954948] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:12,610 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:48:12,613 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:12,618 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:12,618 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:48:12,618 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:48:12,621 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:48:12,622 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#977628211] 01:48:12,627 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:12,633 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from null to Follower 01:48:12,633 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: findLocalShard : cars 01:48:12,633 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-notifier#1227356885] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:12,633 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#1326107190] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:12,634 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testInstallSnapshot with journal batch size 1 01:48:12,635 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:12,635 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:12,635 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 01:48:12,635 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testInstallSnapshot is no longer the leader 01:48:12,635 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testInstallSnapshot , received role change from null to Follower 01:48:12,635 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testInstallSnapshot set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:48:12,635 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Initializing shard [cars] 01:48:12,635 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testInstallSnapshot is initialized 01:48:12,635 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testInstallSnapshot , registered listener akka://cluster-test/user/shardmanager-testInstallSnapshot 01:48:12,635 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-2-shard-cars-testInstallSnapshot from null to Follower 01:48:12,637 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#869779443] 01:48:12,808 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:48:12,808 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Follower): Received ElectionTimeout - switching to Candidate 01:48:12,808 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testInstallSnapshot] 01:48:12,808 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=2, votedFor=member-1-shard-cars-testInstallSnapshot 01:48:12,808 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 2 01:48:12,808 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Follower) :- Switching from behavior Follower to Candidate, election term: 2 01:48:12,808 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-1-shard-cars-testInstallSnapshot (Follower): Closing 01:48:12,808 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 01:48:12,808 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testInstallSnapshot is no longer the leader 01:48:12,808 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from Follower to Candidate 01:48:12,808 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from Follower to Candidate 01:48:13,87 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 01:48:13,87 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=3, votedFor=member-1-shard-cars-testInstallSnapshot 01:48:13,88 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 3 01:48:13,250 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:48:13,397 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 01:48:13,398 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=4, votedFor=member-1-shard-cars-testInstallSnapshot 01:48:13,398 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 4 01:48:13,440 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:13,440 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot 01:48:13,440 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:48:13,440 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:13,440 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:48:13,440 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:48:13,440 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-2-shard-cars-testInstallSnapshot set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:48:13,440 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:48:13,440 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:48:13,452 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:13,452 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot 01:48:13,452 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:48:13,452 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:13,453 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:48:13,453 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:48:13,510 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:48:13,677 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 01:48:13,677 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 01:48:13,678 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 5 01:48:13,678 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] to peer member-2-shard-cars-testInstallSnapshot 01:48:13,683 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:13,683 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Cluster state: [] 01:48:13,683 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] with higher term appears reachable, updating term. 01:48:13,683 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Term 5 in "RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1]" message is greater than follower's term 0 - updating term 01:48:13,683 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=null 01:48:13,683 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): In requestVote: RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 01:48:13,683 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 01:48:13,683 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:48:13,692 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:48:13,692 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:48:13,692 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Election: Leader has following peers: [member-2-shard-cars-testInstallSnapshot] 01:48:13,692 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: false, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 5 01:48:13,692 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:48:13,692 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: true, hasLeader : true 01:48:13,693 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: became leader with frontend state for [] 01:48:13,693 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from Candidate to Leader 01:48:13,693 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testInstallSnapshot, leaderId=member-1-shard-cars-testInstallSnapshot, leaderPayloadVersion=13] 01:48:13,693 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:13,693 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:13,693 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from Candidate to Leader 01:48:13,693 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testInstallSnapshot: All Shards are ready - data store testInstallSnapshot is ready 01:48:13,695 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testInstallSnapshot: Closing 01:48:13,695 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: getLeaderAddress leaderId = member-1-shard-cars-testInstallSnapshot peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:48:13,695 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testInstallSnapshot, leaderId=member-1-shard-cars-testInstallSnapshot, leaderPayloadVersion=13] 01:48:13,695 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:13,695 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:13,695 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot: All Shards are ready - data store testInstallSnapshot is ready 01:48:13,695 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot Received follower initial sync status for member-2-shard-cars-testInstallSnapshot status sync done false 01:48:13,696 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testInstallSnapshot updated: matchIndex: -1, nextIndex: 0 01:48:13,696 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 0, leaderLastIndex: 5, leaderSnapShotIndex: 5 01:48:13,696 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Initiating install snapshot to follower member-2-shard-cars-testInstallSnapshot: follower nextIndex: 0, leader snapshotIndex: 5, leader lastIndex: 5, leader log size: 0 01:48:13,696 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex 5 and lastAppliedTerm 1 instead. 01:48:13,696 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=5, lastAppliedTerm=1, lastIndex=5, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-testInstallSnapshot 01:48:13,696 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: lastSequenceNumber prior to capture: 4 01:48:13,697 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 01:48:13,699 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: CaptureSnapshotReply received by actor 01:48:13,699 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Persisting of snapshot done: Snapshot [lastIndex=5, lastTerm=1, lastAppliedIndex=5, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@46aa74cc, electionTerm=5, electionVotedFor=member-1-shard-cars-testInstallSnapshot, ServerConfigPayload=null] 01:48:13,699 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 5 and term: 1 01:48:13,701 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): sendInstallSnapshot 01:48:13,701 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): next snapshot chunk size for follower member-2-shard-cars-testInstallSnapshot: 376 01:48:13,701 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-testInstallSnapshot, member-2-shard-cars-testInstallSnapshot), Chunk: 1/1 01:48:13,701 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: SaveSnapshotSuccess received for snapshot, sequenceNr: 4 01:48:13,701 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Snapshot success - sequence number: 4 01:48:13,702 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: SnapshotComplete received 01:48:13,702 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot Received follower initial sync status for member-2-shard-cars-testInstallSnapshot status sync done true 01:48:13,702 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 01:48:13,702 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#977628211] 01:48:13,702 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: handled a snapshot response 01:48:13,702 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:48:13,702 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 Queuing TransactionOperation 01:48:13,702 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: handled a journal response 01:48:13,702 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#869779443, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@42493521] 01:48:13,703 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#869779443, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@42493521] 01:48:13,703 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testInstallSnapshot-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#869779443)] for shard cars 01:48:13,703 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 01:48:13,703 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@6702ac12 01:48:13,709 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): handleInstallSnapshot: InstallSnapshot [term=5, leaderId=member-1-shard-cars-testInstallSnapshot, lastIncludedIndex=5, lastIncludedTerm=1, datasize=376, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 01:48:13,710 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 01:48:13,710 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Snapshot installed from leader: member-1-shard-cars-testInstallSnapshot 01:48:13,711 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Applying snapshot on follower: Snapshot [lastIndex=5, lastTerm=1, lastAppliedIndex=5, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@3d6597fe, electionTerm=5, electionVotedFor=member-1-shard-cars-testInstallSnapshot, ServerConfigPayload=null] 01:48:13,711 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - lastSequenceNumber prior to persisting applied snapshot: 2 01:48:13,711 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: SaveSnapshotSuccess received for snapshot, sequenceNr: 2 01:48:13,711 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Snapshot success - sequence number: 2 01:48:13,711 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Moving last applied index from -1 to 5 01:48:13,711 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 01:48:13,711 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Applying snapshot 01:48:13,711 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testInstallSnapshot: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 01:48:13,712 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testInstallSnapshot: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-372705974] with name member-2-shard-cars-testInstallSnapshot-DTCL-publisher 01:48:13,712 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testInstallSnapshot: state snapshot applied in 295.1 μs 01:48:13,712 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Done applying snapshot 01:48:13,713 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 01:48:13,714 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-372705974] 01:48:13,714 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testInstallSnapshot: publishChanges: rootPath: / 01:48:13,714 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testInstallSnapshot: Elapsed time for generation of change events for member-2-shard-cars-testInstallSnapshot-DTCL-publisher: 128.0 μs 01:48:13,714 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-372705974] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:13,714 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=5, followerId=member-2-shard-cars-testInstallSnapshot, chunkIndex=1, success=true] 01:48:13,714 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: SnapshotComplete received 01:48:13,715 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: handled a journal response 01:48:13,715 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: handled a snapshot response 01:48:13,715 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Issuing gracefulStop to shard member-1-shard-cars-testInstallSnapshot 01:48:13,715 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-testInstallSnapshot - waiting on 1 shards 01:48:13,715 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onShutDown 01:48:13,715 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Initiating leader transfer 01:48:13,715 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@45c2107a 01:48:13,715 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testInstallSnapshot: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@45c2107a 01:48:13,715 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Attempting to transfer leadership 01:48:13,716 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testInstallSnapshot, leaderId=null, leaderPayloadVersion=13] 01:48:13,716 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:13,716 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:13,718 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=5, followerId=member-2-shard-cars-testInstallSnapshot, chunkIndex=1, success=true] 01:48:13,718 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Snapshot successfully installed on follower member-2-shard-cars-testInstallSnapshot (last chunk 1) - matchIndex set to 5, nextIndex set to 6 01:48:13,718 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testInstallSnapshot] 01:48:13,718 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testInstallSnapshot, leaderId=null, leaderPayloadVersion=13] 01:48:13,718 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:13,718 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:13,927 PM [cluster-test-shard-dispatcher-25] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testInstallSnapshot: Failed to transfer leadership in 212.1 ms 01:48:13,928 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: In unpauseLeader 01:48:13,928 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: leader transfer failed - sending PoisonPill 01:48:13,928 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Leadership transfer expired 01:48:13,929 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-1-shard-cars-testInstallSnapshot 01:48:13,929 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testInstallSnapshot (Leader)_slicer-id-420: Closing 01:48:13,929 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Aborting 0 pending queued transactions 01:48:13,929 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: All shards shutdown - sending PoisonPill to self 01:48:13,929 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-testInstallSnapshot 01:48:13,935 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntriesReply] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot#-749817139] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#869779443] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#869779443] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:13,935 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 01:48:13,935 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 01:48:13,935 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 01:48:13,935 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: shutting down 01:48:13,936 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 01:48:13,940 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:13,940 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:13,952 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:13,952 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:13,952 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:13,952 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:13,981 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testInstallSnapshot: Closing 01:48:13,981 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testInstallSnapshot 01:48:13,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testInstallSnapshot (Follower): Closing 01:48:13,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Aborting 0 pending queued transactions 01:48:13,982 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testInstallSnapshot 01:48:13,982 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:13,982 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:14,959 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [6175083169199875299] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:15,0 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [2325052457705856564] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:15,2 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:15,3 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:48:15.006] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:15,14 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:15,14 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:15,17 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:15.018] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:15,37 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:15,48 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-2572931785886972617] 01:48:15,48 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:48:15,50 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:48:15,50 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:48:15,50 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:15,50 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:15,61 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:48:15,61 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:48:15,61 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:48:15,66 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:15,77 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-347835135709312354] 01:48:15,78 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:48:15,79 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:15,79 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:48:15,89 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:15,89 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:15,94 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:15,100 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [488336530167555140] 01:48:15,100 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:48:15,101 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:15,101 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:48:15,109 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:15,110 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:15,111 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:15,111 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:15,114 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 01:48:15,145 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 closeTransaction called 01:48:15,145 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:0_7#2094886010)] 01:48:15,145 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 closeTransaction called 01:48:15,145 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:0_6#-1528998637)] 01:48:15,151 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-862935616] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:15,156 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:48:15,190 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 01:48:15,190 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:48:15,190 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@423d142d to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@a9301c4 01:48:15,193 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 01:48:15,193 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:15,193 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:15,193 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:15,200 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1a3986d0 01:48:15,200 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1710164895190, None)) 01:48:15,201 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164895189,0,0)) 01:48:15,202 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@77d9129e 01:48:15,202 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@a9301c4 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@35168f7d 01:48:15,207 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 01:48:15,218 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:15,218 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:15,218 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:15,218 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:15,219 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:15,219 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#736161918] 01:48:15,219 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 01:48:15,220 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:15,221 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 01:48:15,221 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 01:48:15,222 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#736161918] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:15,222 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#1705136407 created and ready for shard:member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:15,223 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-1:snapshot-read#-72801328] 01:48:15,223 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#1705136407] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:15,230 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-1:snapshot-read#-72801328] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:15,239 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:15,239 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:15,239 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Initializing shard [cars] 01:48:15,239 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is initialized 01:48:15,239 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:48:15,239 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 01:48:15,240 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from null to Follower 01:48:15,240 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , registered listener akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader 01:48:15,240 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from null to Follower 01:48:15,247 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-2146927404] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:15,248 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:48:15,254 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:48:15,264 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:15,270 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 01:48:15,270 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:15,270 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 01:48:15,270 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:48:15,270 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@663c82b0 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@88e913e 01:48:15,270 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2ba2ef01 01:48:15,278 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:48:15,290 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1710164895280, None)) 01:48:15,290 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:15,290 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:15,290 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:15,292 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164895279,0,0)) 01:48:15,293 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@88e913e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@634dcacc 01:48:15,293 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1748abdf 01:48:15,302 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 01:48:15,302 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:15,302 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:15,302 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:15,303 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:15,303 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#2091295895] 01:48:15,303 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 01:48:15,303 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#2091295895] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:15,303 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-914535549 created and ready for shard:member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:15,303 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-914535549] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:15,303 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 01:48:15,304 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-2:snapshot-read#-789927854] 01:48:15,304 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-2:snapshot-read#-789927854] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:15,307 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:15,307 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:15,307 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:48:15,307 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 01:48:15,308 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from null to Follower 01:48:15,308 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:15,308 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Initializing shard [cars] 01:48:15,308 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader is initialized 01:48:15,308 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader , registered listener akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader 01:48:15,309 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader from null to Follower 01:48:15,352 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-862935616] 01:48:15,353 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 01:48:15,353 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#736161918] 01:48:15,474 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:48:15,474 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Received ElectionTimeout - switching to Candidate 01:48:15,474 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] 01:48:15,474 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:15,474 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 1 01:48:15,474 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:15,474 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Closing 01:48:15,474 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:48:15,474 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 01:48:15,474 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from Follower to Candidate 01:48:15,474 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from Follower to Candidate 01:48:15,784 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 01:48:15,784 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:15,784 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 2 01:48:15,873 PM [cluster-test-shard-dispatcher-30] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:48:16,54 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 01:48:16,54 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,54 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 3 01:48:16,66 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:48:16,66 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:48:16,67 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:16,67 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,67 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,67 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:16,67 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,67 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,67 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,67 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,130 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:48:16,131 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:16,131 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,131 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,131 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:16,131 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,131 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,131 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,364 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 01:48:16,364 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,364 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 4 01:48:16,364 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,364 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,367 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#736161918] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:16,369 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:16,369 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [] 01:48:16,369 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:48:16,369 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:48:16,369 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=null 01:48:16,369 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:48:16,369 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,369 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:48:16,373 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:48:16,373 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:48:16,373 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Election: Leader has following peers: [member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] 01:48:16,373 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:16,373 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:16,374 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:48:16,374 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:48:16,374 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: became leader with frontend state for [] 01:48:16,374 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from Candidate to Leader 01:48:16,374 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderPayloadVersion=13] 01:48:16,374 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:16,374 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:16,374 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from Candidate to Leader 01:48:16,374 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 01:48:16,376 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#736161918] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:16,380 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Closing 01:48:16,380 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: getLeaderAddress leaderId = member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,380 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader Received follower initial sync status for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader status sync done false 01:48:16,382 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader updated: matchIndex: -1, nextIndex: 0 01:48:16,382 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:16,382 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderPayloadVersion=13] 01:48:16,382 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:16,382 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:16,382 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 01:48:16,416 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:16,419 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:16,422 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 01:48:16,427 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:48:16,427 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1968052177] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:16,429 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 01:48:16,429 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:48:16,429 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@641de9a5 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3de862fc 01:48:16,429 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 01:48:16,430 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@59e5d884 01:48:16,430 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,430 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1710164896429, None)) 01:48:16,430 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164896428,0,0)) 01:48:16,430 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3de862fc to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@500ec018 01:48:16,430 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:16,430 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,430 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 01:48:16,430 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:16,430 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,430 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:16,430 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,430 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@152b90ff 01:48:16,431 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:16,431 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,431 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,431 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,431 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1341570630] 01:48:16,431 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,431 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,431 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1341570630] 01:48:16,431 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:48:16,431 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,431 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,431 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1341570630] 01:48:16,432 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Shard created : member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 01:48:16,432 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-1003624021 created and ready for shard:member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,432 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-3:snapshot-read#-1641467398] 01:48:16,432 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Starting recovery for member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 01:48:16,433 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Stopping Shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:16,433 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 01:48:16,433 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:48:16,434 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:16,434 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:16,440 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:16,440 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:16,440 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:16,440 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:16,455 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-862935616] 01:48:16,455 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 01:48:16,456 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#736161918] 01:48:16,456 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#736161918] 01:48:16,498 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader Received follower initial sync status for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader status sync done true 01:48:17,84 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:48:17,453 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [488336530167555140] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:17,454 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [488336530167555140] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:20,476 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#736161918] to Actor[akka://cluster-test/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:20,476 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#736161918] to Actor[akka://cluster-test/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:20,476 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#736161918] to Actor[akka://cluster-test/deadLetters] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:20,476 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#736161918] to Actor[akka://cluster-test/deadLetters] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:20,477 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#736161918] to Actor[akka://cluster-test/deadLetters] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:20,477 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#736161918] to Actor[akka://cluster-test/deadLetters] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:20,581 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 01:48:21,184 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 01:48:22,190 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 01:48:22,191 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:48:22,193 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:22,215 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:22,222 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 01:48:22,390 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-2146927404] 01:48:22,390 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 01:48:22,390 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#2091295895] 01:48:22,390 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#736161918] 01:48:22,391 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:22,391 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:22,391 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 01:48:22,391 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 01:48:22,391 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 01:48:22,391 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 01:48:22,391 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 01:48:22,391 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 01:48:22,391 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 01:48:22,391 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 01:48:22,391 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:22,391 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:22,391 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 01:48:22,391 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 01:48:22,391 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 01:48:22,391 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 01:48:22,391 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): In requestVote: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 01:48:22,391 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 01:48:22,391 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 01:48:22,391 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 01:48:22,391 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#2091295895] to Actor[akka://cluster-test/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:22,391 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#2091295895] to Actor[akka://cluster-test/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:22,392 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#736161918] 01:48:22,392 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:48:22,392 PM [main] [WARN] ActorUtils - ShardManager for testSemiReachableCandidateNotDroppingLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.testSemiReachableCandidateNotDroppingLeader(DistributedDataStoreRemotingIntegrationTest.java:1322) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:48:22,393 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:48:22,393 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:48:22,393 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Issuing gracefulStop to shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:22,393 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader - waiting on 1 shards 01:48:22,393 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onShutDown 01:48:22,393 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Initiating leader transfer 01:48:22,394 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6313e1de 01:48:22,394 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6313e1de 01:48:22,394 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Attempting to transfer leadership 01:48:22,394 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:22,394 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:22,394 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 01:48:22,394 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:22,394 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:22,395 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader] 01:48:22,396 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 01:48:22,396 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:22,396 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:22,399 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, matchIndex: -1, lastIndex: -1, isVoting: true 01:48:22,399 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Follower's log matches - sending ElectionTimeout 01:48:22,399 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:22,399 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:22,399 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Leader transfer complete 01:48:22,617 PM [cluster-test-shard-dispatcher-12] [WARN] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 215, lastApplied : -1, commitIndex : -1 01:48:22,804 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All shards shutdown - sending PoisonPill to self 01:48:22,804 PM [cluster-test-akka.actor.default-dispatcher-32] [WARN] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#736161918]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:48:22,805 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:22,805 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader)_slicer-id-425: Closing 01:48:22,805 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 01:48:22,805 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:48:22,805 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:48:22,805 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:48:22,806 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: shutting down 01:48:22,806 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:48:22,806 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:48:22,806 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:22,806 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:23,353 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Closing 01:48:23,353 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 01:48:23,823 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-2572931785886972617] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:23,826 PM [cluster-test-akka.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:23,826 PM [cluster-test-akka.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:23,829 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:23.831] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:23,835 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:48:23,836 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:48:23,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Closing 01:48:23,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 01:48:23,836 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:48:23,837 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:23,837 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:23,839 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:23.840] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:23,854 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:23,863 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [3315886629481645290] 01:48:23,863 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:48:23,863 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:48:23,863 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:48:23,873 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:23,873 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:23,874 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:48:23,874 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:48:23,874 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:48:23,882 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:23,891 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-4236384345188919487] 01:48:23,891 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:48:23,893 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:23,893 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:48:23,894 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:23,894 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:23,908 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:23,922 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [8497551983762390724] 01:48:23,922 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:48:23,938 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:23,938 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:48:23,946 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:23,946 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:23,951 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:23,952 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:23,954 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadyLocalTransactionForwardedToLeader 01:48:23,962 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 01:48:23,966 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1814042837] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:23,989 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: no initial generation override, starting from 0 01:48:23,989 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 01:48:23,989 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5373024a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@73b89469 01:48:23,989 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@77870ec6 01:48:23,989 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, 0, 1710164903989, None)) 01:48:23,990 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testReadyLocalTransactionForwardedToLeader 01:48:23,990 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Creating local shard: member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:23,990 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:23,990 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:23,990 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164903988,0,0)) 01:48:23,990 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3fc302f9 01:48:23,991 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@73b89469 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@a386370 01:48:23,991 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} started 01:48:23,991 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:23,993 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:23,994 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:23,994 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:23,995 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadyLocalTransactionForwardedToLeader 01:48:24,6 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,6 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,6 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: updated peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-198402803] 01:48:24,7 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, persistent : true 01:48:24,7 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-198402803] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:24,7 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-1:snapshot-read#-646712311] 01:48:24,7 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with journal batch size 1 01:48:24,7 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#-936717904 created and ready for shard:member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,8 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-1:snapshot-read#-646712311] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:24,9 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:24,9 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:24,9 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Initializing shard [cars] 01:48:24,9 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is initialized 01:48:24,9 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:48:24,9 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 01:48:24,9 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from null to Follower 01:48:24,9 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader 01:48:24,9 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from null to Follower 01:48:24,19 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:48:24,22 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#-936717904] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:24,23 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 01:48:24,26 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-333584524] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:24,31 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:24,47 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:48:24,48 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: no initial generation override, starting from 0 01:48:24,48 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 01:48:24,48 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@805769e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@58b82215 01:48:24,49 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1dbaa766 01:48:24,49 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, 0, 1710164904049, None)) 01:48:24,49 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164904048,0,0)) 01:48:24,49 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@58b82215 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@66c3aaff 01:48:24,50 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} started 01:48:24,50 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadyLocalTransactionForwardedToLeader 01:48:24,50 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Creating local shard: member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,51 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:24,51 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,51 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@20133c52 01:48:24,51 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:24,51 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,51 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,51 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,51 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: updated peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-1408773660] 01:48:24,51 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, persistent : true 01:48:24,52 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with journal batch size 1 01:48:24,52 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-1408773660] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:24,52 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-2:snapshot-read#-652250712] 01:48:24,52 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-2:snapshot-read#-652250712] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:24,53 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:24,53 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:24,53 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:48:24,53 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 01:48:24,53 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Initializing shard [cars] 01:48:24,53 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader is initialized 01:48:24,53 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,61 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#1383801434 created and ready for shard:member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,61 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from null to Follower 01:48:24,61 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader 01:48:24,62 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#1383801434] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:24,62 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader from null to Follower 01:48:24,74 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:24,100 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1814042837] 01:48:24,100 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 01:48:24,100 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-198402803] 01:48:24,308 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:48:24,308 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Received ElectionTimeout - switching to Candidate 01:48:24,308 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] 01:48:24,308 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,308 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 1 01:48:24,308 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:24,308 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Closing 01:48:24,308 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:48:24,308 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 01:48:24,309 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from Follower to Candidate 01:48:24,309 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from Follower to Candidate 01:48:24,584 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:48:24,627 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:48:24,627 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,628 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 2 01:48:24,889 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:48:24,889 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:48:24,890 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:24,890 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,890 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,890 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:24,890 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,890 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,890 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,898 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:24,898 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,898 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:24,898 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,938 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:48:24,938 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,938 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 3 01:48:24,938 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,940 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:24,940 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Cluster state: [] 01:48:24,941 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:48:24,941 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:48:24,941 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=3, votedFor=null 01:48:24,941 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 01:48:24,941 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,941 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 01:48:24,943 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 01:48:24,943 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 01:48:24,943 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] 01:48:24,943 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:24,944 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 01:48:24,944 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:48:24,944 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: became leader with frontend state for [] 01:48:24,944 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from Candidate to Leader 01:48:24,944 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderPayloadVersion=13] 01:48:24,944 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:24,944 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:24,944 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from Candidate to Leader 01:48:24,944 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All Shards are ready - data store testReadyLocalTransactionForwardedToLeader is ready 01:48:24,945 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Closing 01:48:24,945 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:24,945 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderPayloadVersion=13] 01:48:24,945 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:24,945 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:24,945 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All Shards are ready - data store testReadyLocalTransactionForwardedToLeader is ready 01:48:24,945 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader status sync done false 01:48:24,946 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: -1, nextIndex: 0 01:48:24,946 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:24,957 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:48:25,5 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-333584524] 01:48:25,5 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 01:48:25,5 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-1408773660] 01:48:25,5 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:25,5 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-333584524] 01:48:25,5 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 01:48:25,5 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-1408773660] 01:48:25,17 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: handleReadyLocalTransaction for member-1-ShardTransactionTest-fe-0-txn-197-0 01:48:25,17 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:25,17 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Forwarding ReadyLocalTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader)] 01:48:25,20 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, disabling transaction tracking 01:48:25,21 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:48:25,21 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:48:25,21 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:48:25,21 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:25,21 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:25,21 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 01:48:25,21 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-197-0 01:48:25,22 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-197-0 of 1 operations, client version 13 01:48:25,22 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-197-0 01:48:25,22 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-197-0 ready 01:48:25,22 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-197-0 01:48:25,22 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 validated 01:48:25,22 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:25,22 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-197-0: success 01:48:25,22 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-197-0 01:48:25,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-197-0 01:48:25,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 candidate ready 01:48:25,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-197-0: canCommit - candidate rootPath: / 01:48:25,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-197-0: canCommit - messages: [] 01:48:25,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-197-0: preCommit - successfulFromPrevious: [] 01:48:25,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 prepared 01:48:25,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:25,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-197-0 01:48:25,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-197-0 01:48:25,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 197 01:48:25,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=197}] 01:48:25,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=197}] 01:48:25,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-197-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:25,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:25,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:25,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 submitted to persistence 01:48:25,24 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:48:25,24 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=197}] 01:48:25,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 01:48:25,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:48:25,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:48:25,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:48:25,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:48:25,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 1 01:48:25,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:25,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 0, nextIndex: 1 01:48:25,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:48:25,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 0 01:48:25,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from -1 to 0 01:48:25,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:48:25,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 01:48:25,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 01:48:25,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, nanosAgo=6438964} is already disabled 01:48:25,27 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:48:25,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:25,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:25,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=197}]]] 01:48:25,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 01:48:25,27 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader status sync done true 01:48:25,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=197}]]] 01:48:25,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:48:25,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:48:25,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log Simple{size=197} 01:48:25,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=197}] 01:48:25,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 2 01:48:25,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 0 01:48:25,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:25,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:48:25,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 0 01:48:25,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from -1 to 0 01:48:25,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:48:25,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 01:48:25,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 01:48:25,34 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=197}] 01:48:25,34 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 01:48:25,34 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 1, nextIndex: 2 01:48:25,34 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:48:25,34 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 1 01:48:25,34 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 0 to 1 01:48:25,34 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 01:48:25,34 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-197-0 01:48:25,34 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-197-0: commit - successfulFromPrevious: [] 01:48:25,34 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 changing state from COMMIT_PENDING to COMMITTED 01:48:25,34 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/system/testProbe-761#1920023704] 01:48:25,34 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-1055115473] with name member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher 01:48:25,35 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 3 01:48:25,35 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:25,35 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 01:48:25,35 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-1055115473] 01:48:25,35 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 01:48:25,35 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-1055115473] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:25,36 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 84.76 μs 01:48:25,37 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 01:48:25,37 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1814042837] 01:48:25,37 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:48:25,37 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 Queuing TransactionOperation 01:48:25,37 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-198402803, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@42101d1d] 01:48:25,37 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-198402803, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@42101d1d] 01:48:25,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 1 01:48:25,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:48:25,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 1 01:48:25,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 0 to 1 01:48:25,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 01:48:25,37 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-198402803)] for shard cars 01:48:25,37 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 01:48:25,37 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4e33a70 01:48:25,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-197-0 01:48:25,38 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-1319262440] with name member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher 01:48:25,38 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 3 01:48:25,38 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 01:48:25,38 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-1319262440] 01:48:25,38 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 01:48:25,39 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-1319262440] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:25,39 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 119.6 μs 01:48:25,39 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: handleReadyLocalTransaction for member-1-ShardTransactionTest-fe-0-txn-198-0 01:48:25,39 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:25,39 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Forwarding ReadyLocalTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader)] 01:48:25,43 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-198-0 01:48:25,43 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-198-0 of 1 operations, client version 13 01:48:25,43 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-198-0 01:48:25,43 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-198-0 ready 01:48:25,45 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0 canCommit 01:48:25,45 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-198402803)] resolved 01:48:25,45 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: successfully resolved all cohort actors 01:48:25,45 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0 finishCanCommit 01:48:25,45 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: sending CanCommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-198-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-198402803)] 01:48:25,45 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-198402803)] 01:48:25,46 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Can committing transaction member-1-ShardTransactionTest-fe-0-txn-198-0 01:48:25,46 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-198-0 01:48:25,46 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 validated 01:48:25,46 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:25,46 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-198-0: success 01:48:25,46 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:48:25,46 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: canCommit returning result: true 01:48:25,46 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0 commit 01:48:25,46 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0 finish commit 01:48:25,46 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: Sending CommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-198-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-198402803)] 01:48:25,46 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-198402803)] 01:48:25,46 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-198-0 01:48:25,46 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-198-0 01:48:25,46 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 candidate ready 01:48:25,46 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-198-0: canCommit - candidate rootPath: / 01:48:25,46 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-198-0: canCommit - messages: [] 01:48:25,46 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-198-0: preCommit - successfulFromPrevious: [] 01:48:25,46 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 prepared 01:48:25,46 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:25,46 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-198-0 01:48:25,46 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-198-0 01:48:25,47 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 208 01:48:25,47 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=208}] 01:48:25,47 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=208}] 01:48:25,47 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-198-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:25,47 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:48:25,47 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:25,47 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=208}]]] 01:48:25,47 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 submitted to persistence 01:48:25,48 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=208}] 01:48:25,49 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=208}]]] 01:48:25,49 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:48:25,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:48:25,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log Simple{size=208} 01:48:25,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=208}] 01:48:25,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 2 01:48:25,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=2, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:25,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=208}] 01:48:25,51 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 2, nextIndex: 3 01:48:25,51 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:48:25,51 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 2 01:48:25,51 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 1 to 2 01:48:25,51 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 01:48:25,51 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-198-0 01:48:25,51 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-198-0: commit - successfulFromPrevious: [] 01:48:25,51 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 changing state from COMMIT_PENDING to COMMITTED 01:48:25,51 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testReadyLocalTransactionForwardedToLeader_member-1-shard-cars-testReadyLocalTransactionForwardedToLeader%23-198402803$a] 01:48:25,51 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: commit succeeded 01:48:25,51 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 01:48:25,51 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 3 01:48:25,51 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:48:25,51 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 112.5 μs 01:48:25,51 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 01:48:25,51 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0 01:48:25,52 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-198402803)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:25,52 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4e33a70 01:48:25,52 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:25,52 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 01:48:25,52 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Issuing gracefulStop to shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:25,52 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader - waiting on 1 shards 01:48:25,52 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onShutDown 01:48:25,52 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Initiating leader transfer 01:48:25,52 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4602a2ae 01:48:25,52 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4602a2ae 01:48:25,52 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Attempting to transfer leadership 01:48:25,53 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:48:25,53 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 01:48:25,53 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:25,53 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:25,55 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 2 01:48:25,55 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:48:25,55 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 2 01:48:25,55 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 1 to 2 01:48:25,55 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 01:48:25,56 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-198-0 01:48:25,56 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 3 01:48:25,57 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 01:48:25,57 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader] 01:48:25,57 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 01:48:25,57 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 01:48:25,57 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:25,57 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:25,57 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 106.2 μs 01:48:25,58 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, matchIndex: 2, lastIndex: 2, isVoting: true 01:48:25,58 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Follower's log matches - sending ElectionTimeout 01:48:25,58 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:48:25,58 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Leader transfer complete 01:48:25,468 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 01:48:25,468 PM [cluster-test-akka.actor.default-dispatcher-15] [WARN] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-198402803]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:48:25,468 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:25,468 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader)_slicer-id-432: Closing 01:48:25,468 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Aborting 0 pending queued transactions 01:48:25,469 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 01:48:25,469 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 01:48:25,469 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 01:48:25,469 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 01:48:25,469 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: shutting down 01:48:25,469 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 01:48:25,470 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1814042837] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1814042837] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:25,470 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1814042837] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1814042837] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:25,470 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1814042837] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1814042837] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:25,470 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1814042837] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1814042837] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:25,471 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:25,471 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:25,479 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:25,479 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:25,479 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:25,480 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:25,482 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:48:25,499 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 01:48:25,499 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:48:25,499 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Closing 01:48:25,499 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Aborting 0 pending queued transactions 01:48:25,499 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 01:48:25,513 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:25,513 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:26,487 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [3315886629481645290] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:26,517 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-4236384345188919487] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:26,518 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:26,518 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:26,520 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:26.523] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:26,537 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:26,537 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:26,540 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:26.541] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:26,552 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:48:26,560 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [9098709125816946519] 01:48:26,560 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:48:26,561 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:48:26,561 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:48:26,562 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:26,562 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:26,571 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:48:26,571 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:48:26,571 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:48:26,576 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:26,581 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [5538902800617192666] 01:48:26,582 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:48:26,583 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:26,583 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:48:26,589 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:26,589 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:26,611 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:26,621 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [6744781546911783421] 01:48:26,623 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:48:26,624 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:26,624 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:48:26,630 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:48:26,635 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:26,637 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:26,638 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithSingleShard 01:48:26,641 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:26,641 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:26,648 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#1672506387] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:26,649 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:26,676 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithSingleShard 01:48:26,677 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:48:26,681 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:26,687 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: no initial generation override, starting from 0 01:48:26,687 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:48:26,687 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithSingleShard 01:48:26,687 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6b2616f5 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@325cb2cb 01:48:26,687 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Creating local shard: member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:48:26,687 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7dfc224d 01:48:26,687 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:26,687 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, 0, 1710164906687, None)) 01:48:26,687 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:48:26,687 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164906686,0,0)) 01:48:26,688 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@325cb2cb to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@74d1f063 01:48:26,688 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} started 01:48:26,688 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:26,690 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:26,691 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithSingleShard 01:48:26,692 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5135ae0c 01:48:26,692 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:26,692 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:48:26,692 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:48:26,692 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:48:26,692 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testReadWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard on shard member-1-shard-cars-testReadWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-1129651698] 01:48:26,693 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testReadWriteTransactionWithSingleShard, persistent : true 01:48:26,693 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-1129651698] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:26,693 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-notifier#1866830617 created and ready for shard:member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:48:26,693 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-1:snapshot-read#-1633204629] 01:48:26,693 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-notifier#1866830617] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:26,693 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadWriteTransactionWithSingleShard with journal batch size 1 01:48:26,693 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-1:snapshot-read#-1633204629] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:26,697 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:26,697 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:26,697 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Initializing shard [cars] 01:48:26,697 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is initialized 01:48:26,697 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:48:26,697 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 01:48:26,697 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from null to Follower 01:48:26,697 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard 01:48:26,697 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from null to Follower 01:48:26,698 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithSingleShard 01:48:26,706 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-1820663240] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:26,708 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: no initial generation override, starting from 0 01:48:26,708 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:48:26,708 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@20c05495 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@59ba4af3 01:48:26,708 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@15c05c7d 01:48:26,709 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, 0, 1710164906708, None)) 01:48:26,709 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithSingleShard 01:48:26,709 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:48:26,709 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:26,709 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:48:26,709 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164906707,0,0)) 01:48:26,710 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@59ba4af3 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5c4f14c1 01:48:26,710 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} started 01:48:26,710 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@715440ef 01:48:26,710 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:26,710 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:48:26,710 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:48:26,710 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:48:26,710 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testReadWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard on shard member-2-shard-cars-testReadWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard#703789795] 01:48:26,711 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testReadWriteTransactionWithSingleShard, persistent : true 01:48:26,711 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadWriteTransactionWithSingleShard with journal batch size 1 01:48:26,711 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard#703789795] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:26,711 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-822291723 created and ready for shard:member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:48:26,711 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-1747941644] 01:48:26,712 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-822291723] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:26,712 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-1747941644] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:26,712 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:26,712 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:26,712 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:48:26,712 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 01:48:26,712 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:48:26,712 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Initializing shard [cars] 01:48:26,712 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard is initialized 01:48:26,713 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithSingleShard , received role change from null to Follower 01:48:26,713 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard 01:48:26,713 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testReadWriteTransactionWithSingleShard from null to Follower 01:48:26,760 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#1672506387] 01:48:26,760 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: findLocalShard : cars 01:48:26,761 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-1129651698] 01:48:26,940 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:48:26,940 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 01:48:26,940 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadWriteTransactionWithSingleShard] 01:48:26,940 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:48:26,941 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 1 01:48:26,941 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:26,941 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Closing 01:48:26,941 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:48:26,941 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 01:48:26,941 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from Follower to Candidate 01:48:26,941 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from Follower to Candidate 01:48:27,240 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:48:27,240 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:48:27,240 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 2 01:48:27,263 PM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:48:27,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:48:27,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:48:27,510 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 3 01:48:27,582 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:27,582 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard 01:48:27,582 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:48:27,582 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:27,582 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:48:27,582 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:48:27,582 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:48:27,582 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:48:27,582 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:48:27,596 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:27,596 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard 01:48:27,596 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:48:27,596 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:27,596 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:48:27,596 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:48:27,646 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:48:27,830 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:48:27,830 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:48:27,830 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 4 01:48:27,831 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:48:27,832 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:27,832 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Cluster state: [] 01:48:27,832 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:48:27,832 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:48:27,832 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 01:48:27,832 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:48:27,832 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:48:27,833 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:48:27,834 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:48:27,834 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:48:27,834 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadWriteTransactionWithSingleShard] 01:48:27,834 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:27,834 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:48:27,834 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:48:27,834 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: became leader with frontend state for [] 01:48:27,835 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from Candidate to Leader 01:48:27,835 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:48:27,835 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:27,835 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:27,835 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from Candidate to Leader 01:48:27,835 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All Shards are ready - data store testReadWriteTransactionWithSingleShard is ready 01:48:27,835 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Closing 01:48:27,835 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testReadWriteTransactionWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:48:27,836 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:48:27,836 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:27,836 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:27,836 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All Shards are ready - data store testReadWriteTransactionWithSingleShard is ready 01:48:27,836 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithSingleShard status sync done false 01:48:27,836 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: -1, nextIndex: 0 01:48:27,836 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:27,871 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testReadWriteTransactionWithSingleShard commit rateLimit adjusted to 176.34356011097387 01:48:27,871 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:27,872 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-1820663240] 01:48:27,872 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:48:27,872 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:48:27,872 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:48:27,872 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard, primaryVersion=13] 01:48:27,872 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:48:27,872 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:48:27,872 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard, primaryVersion=13] 01:48:27,872 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] for shard cars 01:48:27,872 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] found - trying create transaction 01:48:27,872 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] 01:48:27,874 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, disabling transaction tracking 01:48:27,874 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 01:48:27,874 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 01:48:27,874 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:48:27,874 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:27,874 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:48:27,874 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}]]] 01:48:27,874 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:27,875 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-497399209] 01:48:27,876 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-497399209] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:27,876 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 01:48:27,878 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-497399209, transactionId=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0, version=13] 01:48:27,878 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:48:27,878 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:48:27,878 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}]]] 01:48:27,878 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 01:48:27,878 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:48:27,878 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 01:48:27,878 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66} 01:48:27,878 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 01:48:27,878 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:48:27,878 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 sending 3 batched modifications, ready: false 01:48:27,878 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-497399209)] 01:48:27,878 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Log size is now 1 01:48:27,878 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:27,878 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-497399209)] 01:48:27,879 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 01:48:27,879 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithSingleShard status sync done true 01:48:27,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: 0, nextIndex: 1 01:48:27,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:48:27,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Setting last applied to 0 01:48:27,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 01:48:27,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66} 01:48:27,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:48:27,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadWriteTransactionWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} does not match any client, pre-disabling client. 01:48:27,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, nanosAgo=7139086} is already disabled 01:48:27,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:27,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 01:48:27,884 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 modifications completed with BatchedModificationsReply [numBatched=3] 01:48:27,884 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 ReadData operation succeeded 01:48:27,884 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 01:48:27,884 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeRead DataExists called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 01:48:27,884 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 sending 1 batched modifications, ready: false 01:48:27,884 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-497399209)] 01:48:27,884 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.DataExists to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-497399209)] 01:48:27,884 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Commit index set to 0 01:48:27,884 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:48:27,885 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Setting last applied to 0 01:48:27,885 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 01:48:27,885 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66} 01:48:27,885 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:48:27,885 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadWriteTransactionWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} does not match any client, pre-disabling client. 01:48:27,885 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 01:48:27,888 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 modifications completed with BatchedModificationsReply [numBatched=1] 01:48:27,888 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 DataExists operation succeeded 01:48:27,888 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Readying 1 components for commit 01:48:27,888 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Readying transaction for shard cars 01:48:27,888 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 directCommit called 01:48:27,888 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 sending 0 batched modifications, ready: true 01:48:27,888 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-497399209)] 01:48:27,888 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 canCommit 01:48:27,889 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:27,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: handleForwardedReadyTransaction for member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:27,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Readying transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0, client version 13 01:48:27,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: readying transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:27,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 ready 01:48:27,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:27,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 validated 01:48:27,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:27,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: canCommit for member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: success 01:48:27,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Committing transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:27,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:27,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 candidate ready 01:48:27,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:48:27,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: canCommit - messages: [] 01:48:27,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:48:27,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 prepared 01:48:27,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:27,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Finishing commit for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:27,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:27,890 PM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 249 01:48:27,890 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 01:48:27,890 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 01:48:27,890 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:27,890 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:27,890 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:48:27,890 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}]]] 01:48:27,890 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 submitted to persistence 01:48:27,890 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 01:48:27,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}]]] 01:48:27,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 01:48:27,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:48:27,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=249} 01:48:27,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 01:48:27,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Log size is now 2 01:48:27,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:27,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 01:48:27,906 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: 1, nextIndex: 2 01:48:27,906 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:48:27,906 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Setting last applied to 1 01:48:27,906 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 01:48:27,906 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 1 data Simple{size=249} 01:48:27,906 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:27,906 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:48:27,906 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:48:27,906 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testReadWriteTransactionWithSingleShard_member-1-shard-cars-testReadWriteTransactionWithSingleShard_shard-cars-member-2%3Adatastore-testReadWriteTransactionWithSingleShard%400%3A0_8%23-497399209$f] 01:48:27,906 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-416177456] with name member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher 01:48:27,907 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 01:48:27,907 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:27,907 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 01:48:27,908 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-416177456] 01:48:27,908 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:48:27,908 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-416177456] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:27,908 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher: 121.6 μs 01:48:27,912 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 modifications completed with CommitTransactionReply [version=13] 01:48:27,912 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 successfully completed direct commit 01:48:27,912 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 01:48:27,912 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Commit index set to 1 01:48:27,912 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:48:27,912 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Setting last applied to 1 01:48:27,912 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:27,913 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 01:48:27,913 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:48:27,913 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 1 data Simple{size=249} 01:48:27,913 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] found - trying create transaction 01:48:27,913 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] 01:48:27,913 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 Queuing TransactionOperation 01:48:27,913 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:27,913 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-39557101] with name member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher 01:48:27,914 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 01:48:27,914 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 01:48:27,914 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-39557101] 01:48:27,914 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:48:27,914 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-39557101] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:27,915 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher: 102.9 μs 01:48:27,916 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 01:48:27,916 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_9#1184037765] 01:48:27,917 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_9#1184037765] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:27,917 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_9#1184037765, transactionId=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0, version=13] 01:48:27,918 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:48:27,918 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_9#1184037765)] 01:48:27,920 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 ReadData operation succeeded 01:48:27,920 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:48:27,920 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:48:27,920 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithSingleShard - waiting on 1 shards 01:48:27,921 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onShutDown 01:48:27,921 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Initiating leader transfer 01:48:27,921 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@71ddc47 01:48:27,921 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@71ddc47 01:48:27,921 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Attempting to transfer leadership 01:48:27,921 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:27,921 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:48:27,921 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:27,921 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:27,922 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard] 01:48:27,922 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:48:27,922 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:27,922 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:27,927 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadWriteTransactionWithSingleShard, matchIndex: 1, lastIndex: 1, isVoting: true 01:48:27,927 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 01:48:27,927 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:27,927 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Leader transfer complete 01:48:28,340 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 01:48:28,340 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-1129651698]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:48:28,341 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:48:28,341 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader)_slicer-id-437: Closing 01:48:28,341 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:48:28,341 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithSingleShard 01:48:28,341 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 01:48:28,341 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:48:28,342 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#1672506387] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#1672506387] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:28,342 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 01:48:28,342 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: shutting down 01:48:28,342 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 01:48:28,342 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#1672506387] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#1672506387] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:28,342 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#1672506387] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#1672506387] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:28,343 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:28,343 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:28,355 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:28,355 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:28,355 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:28,355 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:28,358 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:48:28,373 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:48:28,373 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 01:48:28,373 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Closing 01:48:28,374 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:48:28,374 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithSingleShard 01:48:28,375 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:28,375 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:29,366 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [9098709125816946519] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:29,394 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:29,394 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:29,395 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [5538902800617192666] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [03/11/2024 13:48:29.397] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:29,415 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:29,415 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:29,418 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:29.419] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:29,433 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:29,439 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [1038413523242514956] 01:48:29,439 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:48:29,440 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:48:29,440 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:48:29,450 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:29,450 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:29,453 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:48:29,453 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:48:29,453 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:48:29,461 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:29,477 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-4382350408938990469] 01:48:29,477 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:48:29,477 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:29,477 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:48:29,477 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:29,477 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:29,492 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:29,503 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-5796962388653744919] 01:48:29,503 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:48:29,509 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:29,509 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:48:29,509 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:29,509 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:29,522 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 closeTransaction called 01:48:29,522 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_9#1184037765)] 01:48:29,525 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:29,527 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:29,528 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSnapshotOnRootOverwrite 01:48:29,533 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testSnapshotOnRootOverwrite 01:48:29,535 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-525977620] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:29,541 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:48:29,544 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:29,548 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testSnapshotOnRootOverwrite 01:48:29,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-1-shard-cars-testSnapshotOnRootOverwrite 01:48:29,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-1-shard-default-testSnapshotOnRootOverwrite 01:48:29,548 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: no initial generation override, starting from 0 01:48:29,548 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:48:29,548 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@64114b46 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@27e0a771 01:48:29,548 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:29,548 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@23df3755 01:48:29,548 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:48:29,548 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:48:29,549 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSnapshotOnRootOverwrite, 0, 1710164909548, None)) 01:48:29,549 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164909547,0,0)) 01:48:29,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@34a2981e 01:48:29,549 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@27e0a771 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5771e93f 01:48:29,549 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} started 01:48:29,550 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:29,551 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:29,551 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSnapshotOnRootOverwrite 01:48:29,552 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:29,555 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSnapshotOnRootOverwrite 01:48:29,557 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:48:29,557 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:48:29,557 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-cars-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite on shard member-1-shard-cars-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842] 01:48:29,557 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-testSnapshotOnRootOverwrite 01:48:29,558 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:48:29,558 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:48:29,558 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-default-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite on shard member-1-shard-default-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027] 01:48:29,558 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-1-shard-cars-testSnapshotOnRootOverwrite, persistent : true 01:48:29,558 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-default-testSnapshotOnRootOverwrite, persistent : true 01:48:29,559 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSnapshotOnRootOverwrite with journal batch size 1 01:48:29,559 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:29,559 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-default-testSnapshotOnRootOverwrite with journal batch size 1 01:48:29,559 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:29,559 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-notifier#489349338 created and ready for shard:member-1-shard-cars-testSnapshotOnRootOverwrite 01:48:29,559 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-248358058] 01:48:29,559 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-1939895755] 01:48:29,559 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-notifier#1082302933] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:29,559 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-notifier#489349338] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:29,559 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-248358058] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:29,559 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-1939895755] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:29,560 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-notifier#1082302933 created and ready for shard:member-1-shard-default-testSnapshotOnRootOverwrite 01:48:29,563 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:29,563 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:29,564 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [cars] 01:48:29,564 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSnapshotOnRootOverwrite is initialized 01:48:29,566 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:29,566 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:29,566 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:48:29,566 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-default-testSnapshotOnRootOverwrite is no longer the leader 01:48:29,566 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [default] 01:48:29,566 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-testSnapshotOnRootOverwrite is initialized 01:48:29,568 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:48:29,568 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 01:48:29,568 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from null to Follower 01:48:29,568 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 01:48:29,568 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from null to Follower 01:48:29,568 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 01:48:29,570 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:48:29,572 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:29,574 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from null to Follower 01:48:29,574 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from null to Follower 01:48:29,577 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1897809026] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:29,577 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testSnapshotOnRootOverwrite 01:48:29,579 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: no initial generation override, starting from 0 01:48:29,579 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:48:29,579 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@64791213 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@44e24b7a 01:48:29,579 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testSnapshotOnRootOverwrite 01:48:29,579 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:29,579 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-2-shard-default-testSnapshotOnRootOverwrite 01:48:29,580 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:29,580 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:48:29,580 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:48:29,580 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@56e5c613 01:48:29,580 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSnapshotOnRootOverwrite, 0, 1710164909580, None)) 01:48:29,580 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164909579,0,0)) 01:48:29,580 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@44e24b7a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@462ea6ad 01:48:29,580 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} started 01:48:29,581 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2ed3993c 01:48:29,581 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:29,581 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:29,581 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:48:29,581 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:29,581 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-cars-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite on shard member-2-shard-cars-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-47345733] 01:48:29,581 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-default-testSnapshotOnRootOverwrite 01:48:29,582 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testSnapshotOnRootOverwrite, persistent : true 01:48:29,582 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSnapshotOnRootOverwrite with journal batch size 1 01:48:29,582 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-47345733] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:29,582 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-default-testSnapshotOnRootOverwrite, persistent : true 01:48:29,582 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#495997100] 01:48:29,583 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-notifier#-718372079 created and ready for shard:member-2-shard-default-testSnapshotOnRootOverwrite 01:48:29,583 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-notifier#-1590618950] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:29,584 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:48:29,584 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:48:29,584 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-default-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite on shard member-2-shard-default-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite#-599498697] 01:48:29,584 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-notifier#-1590618950 created and ready for shard:member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:29,584 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#-1621698726] 01:48:29,584 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-default-testSnapshotOnRootOverwrite with journal batch size 1 01:48:29,585 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite#-599498697] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:29,585 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#495997100] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:29,585 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-notifier#-718372079] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:29,585 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#-1621698726] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:29,586 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:29,586 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:29,586 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:48:29,586 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-2-shard-default-testSnapshotOnRootOverwrite is no longer the leader 01:48:29,586 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-default-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:48:29,586 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [default] 01:48:29,586 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-default-testSnapshotOnRootOverwrite is initialized 01:48:29,586 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-default-testSnapshotOnRootOverwrite , received role change from null to Follower 01:48:29,586 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:29,586 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:29,586 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:48:29,586 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 01:48:29,586 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:48:29,586 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-default-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 01:48:29,586 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSnapshotOnRootOverwrite , received role change from null to Follower 01:48:29,587 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [cars] 01:48:29,587 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSnapshotOnRootOverwrite is initialized 01:48:29,587 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-2-shard-default-testSnapshotOnRootOverwrite from null to Follower 01:48:29,587 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 01:48:29,587 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-2-shard-cars-testSnapshotOnRootOverwrite from null to Follower 01:48:29,631 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-525977620] 01:48:29,631 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 01:48:29,631 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842] 01:48:29,820 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:48:29,820 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Received ElectionTimeout - switching to Candidate 01:48:29,820 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSnapshotOnRootOverwrite] 01:48:29,820 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=1, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 01:48:29,821 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 1 01:48:29,821 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:29,821 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Closing 01:48:29,821 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:48:29,821 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 01:48:29,821 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from Follower to Candidate 01:48:29,821 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from Follower to Candidate 01:48:29,830 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:48:29,830 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Received ElectionTimeout - switching to Candidate 01:48:29,830 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Election: Candidate has following voting peers: [member-2-shard-default-testSnapshotOnRootOverwrite] 01:48:29,830 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=1, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 01:48:29,830 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 1 01:48:29,831 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:29,831 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Closing 01:48:29,831 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:48:29,831 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-default-testSnapshotOnRootOverwrite is no longer the leader 01:48:29,831 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from Follower to Candidate 01:48:29,831 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from Follower to Candidate 01:48:30,50 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:48:30,51 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=2, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 01:48:30,51 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 2 01:48:30,124 PM [cluster-test-shard-dispatcher-33] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:48:30,141 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:48:30,141 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=2, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 01:48:30,141 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 2 01:48:30,184 PM [cluster-test-shard-dispatcher-33] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:48:30,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:48:30,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 01:48:30,281 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 3 01:48:30,391 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:48:30,391 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 01:48:30,391 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 3 01:48:30,462 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:30,462 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite 01:48:30,462 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:48:30,462 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite 01:48:30,462 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:48:30,462 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:30,462 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:30,462 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:48:30,462 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:48:30,462 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:48:30,462 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:30,462 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - Peer address for peer member-2-shard-default-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:48:30,462 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:48:30,462 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:48:30,471 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:30,471 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite 01:48:30,471 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:30,471 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite 01:48:30,471 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:48:30,471 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:30,471 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:30,471 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:30,471 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:48:30,471 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:48:30,524 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:48:30,590 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:48:30,590 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 01:48:30,590 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 4 01:48:30,591 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-default-testSnapshotOnRootOverwrite 01:48:30,592 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:30,592 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Cluster state: [] 01:48:30,592 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:48:30,592 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:48:30,592 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=null 01:48:30,592 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:48:30,592 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 01:48:30,592 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:48:30,594 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:48:30,594 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:48:30,594 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Election: Leader has following peers: [member-2-shard-default-testSnapshotOnRootOverwrite] 01:48:30,594 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:30,594 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:48:30,594 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: true, hasLeader : true 01:48:30,594 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: became leader with frontend state for [] 01:48:30,594 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from Candidate to Leader 01:48:30,594 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 01:48:30,594 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 01:48:30,595 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from Candidate to Leader 01:48:30,595 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 01:48:30,596 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-default-testSnapshotOnRootOverwrite: Closing 01:48:30,596 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: getLeaderAddress leaderId = member-1-shard-default-testSnapshotOnRootOverwrite peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:48:30,597 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-default-testSnapshotOnRootOverwrite status sync done false 01:48:30,598 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: -1, nextIndex: 0 01:48:30,598 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:30,599 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-testSnapshotOnRootOverwrite, leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 01:48:30,599 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 01:48:30,599 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 01:48:30,670 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:48:30,670 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 01:48:30,670 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 4 01:48:30,671 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:30,672 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:30,673 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Cluster state: [] 01:48:30,673 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:48:30,673 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:48:30,673 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=null 01:48:30,673 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:48:30,673 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 01:48:30,673 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:48:30,675 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:48:30,675 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:48:30,675 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Election: Leader has following peers: [member-2-shard-cars-testSnapshotOnRootOverwrite] 01:48:30,675 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:30,675 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:48:30,675 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: true, hasLeader : true 01:48:30,675 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: became leader with frontend state for [] 01:48:30,675 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from Candidate to Leader 01:48:30,675 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 01:48:30,675 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:30,675 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:30,675 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from Candidate to Leader 01:48:30,675 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All Shards are ready - data store testSnapshotOnRootOverwrite is ready 01:48:30,677 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-cars-testSnapshotOnRootOverwrite: Closing 01:48:30,677 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: getLeaderAddress leaderId = member-1-shard-cars-testSnapshotOnRootOverwrite peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:48:30,677 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-cars-testSnapshotOnRootOverwrite status sync done false 01:48:30,677 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSnapshotOnRootOverwrite, leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 01:48:30,677 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:30,677 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:30,678 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All Shards are ready - data store testSnapshotOnRootOverwrite is ready 01:48:30,678 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: -1, nextIndex: 0 01:48:30,678 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:30,712 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-default-testSnapshotOnRootOverwrite status sync done true 01:48:30,746 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-525977620] 01:48:30,746 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : default 01:48:30,747 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027] 01:48:30,748 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testSnapshotOnRootOverwrite commit rateLimit adjusted to 40.88563184859886 01:48:30,748 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,749 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-525977620] 01:48:30,749 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:48:30,749 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@38a39cd1] 01:48:30,750 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@38a39cd1] 01:48:30,750 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars 01:48:30,750 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 01:48:30,750 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:30,750 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-525977620] 01:48:30,750 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:48:30,750 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@297a6dae] 01:48:30,750 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@297a6dae] 01:48:30,750 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027)] for shard default 01:48:30,750 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Shard default resolved to local data tree - adding local factory 01:48:30,750 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 - Creating local component for shard default using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@59b9c916 01:48:30,751 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 Readying 2 components for commit 01:48:30,751 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 Readying transaction for shard cars 01:48:30,751 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 = [ 1 ] 01:48:30,751 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:30,751 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 Readying transaction for shard default 01:48:30,751 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 = [ 1 ] 01:48:30,751 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,751 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027)] 01:48:30,751 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, disabling transaction tracking 01:48:30,751 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 canCommit 01:48:30,751 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 01:48:30,751 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 01:48:30,751 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:48:30,751 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:30,751 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:30,751 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}]]] 01:48:30,752 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,752 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,752 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, disabling transaction tracking 01:48:30,752 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 01:48:30,752 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 01:48:30,752 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:48:30,752 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:30,752 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 01:48:30,752 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}]]] 01:48:30,752 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransactionReadyReplyMapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 readyTransaction 01:48:30,752 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,752 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 01:48:30,752 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] TransactionReadyReplyMapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 readyTransaction 01:48:30,752 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] resolved 01:48:30,752 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 resolved to actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:30,752 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027)] resolved 01:48:30,752 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: successfully resolved all cohort actors 01:48:30,752 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 finishCanCommit 01:48:30,753 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:30,753 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:30,753 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 resolved to actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027)] 01:48:30,753 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}]]] 01:48:30,753 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:48:30,753 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:48:30,753 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 01:48:30,753 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 01:48:30,754 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 1 01:48:30,754 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:30,754 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}]]] 01:48:30,754 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:48:30,754 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:48:30,754 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 01:48:30,754 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 01:48:30,754 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 1 01:48:30,754 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:30,754 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 01:48:30,754 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 01:48:30,755 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Can committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,755 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,755 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 validated 01:48:30,755 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:30,755 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: success 01:48:30,755 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-cars-testSnapshotOnRootOverwrite status sync done true 01:48:30,755 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:48:30,755 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 01:48:30,755 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027)] 01:48:30,755 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027)] 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 0, nextIndex: 1 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 0 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendMetadata - member-1-shard-default-testSnapshotOnRootOverwrite: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} does not match any client, pre-disabling client. 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, nanosAgo=8153643} is already disabled 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Can committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 validated 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: success 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 0, nextIndex: 1 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 0 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:48:30,760 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendMetadata - member-1-shard-cars-testSnapshotOnRootOverwrite: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} does not match any client, pre-disabling client. 01:48:30,760 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit returning result: true 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, nanosAgo=9409243} is already disabled 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:30,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 01:48:30,761 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 commit 01:48:30,761 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 finish commit 01:48:30,761 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:30,761 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:30,761 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027)] 01:48:30,761 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027)] 01:48:30,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 candidate ready 01:48:30,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:48:30,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - messages: [] 01:48:30,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:48:30,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 prepared 01:48:30,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:30,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 113 01:48:30,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] 01:48:30,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] 01:48:30,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:30,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:30,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 01:48:30,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}]]] 01:48:30,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 submitted to persistence 01:48:30,761 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,761 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,761 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 candidate ready 01:48:30,762 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:48:30,762 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - messages: [] 01:48:30,762 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:48:30,762 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 prepared 01:48:30,762 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:30,762 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,762 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,762 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] 01:48:30,762 PM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 282 01:48:30,762 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] 01:48:30,762 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] 01:48:30,762 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:30,762 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:30,762 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:30,762 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}]]] 01:48:30,762 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 submitted to persistence 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 0 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 0 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendMetadata - member-2-shard-default-testSnapshotOnRootOverwrite: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} does not match any client, pre-disabling client. 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}]]] 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=113} 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 0 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 0 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendMetadata - member-2-shard-cars-testSnapshotOnRootOverwrite: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} does not match any client, pre-disabling client. 01:48:30,763 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 01:48:30,764 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] 01:48:30,764 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 01:48:30,768 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] 01:48:30,768 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}]]] 01:48:30,768 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:48:30,768 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:48:30,768 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=282} 01:48:30,768 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] 01:48:30,768 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:48:30,768 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:30,769 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 01:48:30,769 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 1, nextIndex: 2 01:48:30,769 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:48:30,769 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 1 01:48:30,769 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 01:48:30,769 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 1 data Simple{size=113} 01:48:30,769 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,769 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:48:30,769 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:48:30,769 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-default-testSnapshotOnRootOverwrite%231227224027$a] 01:48:30,769 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-default-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#1267922237] with name member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher 01:48:30,769 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#1267922237] 01:48:30,769 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:48:30,769 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:30,769 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#1267922237] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:30,769 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 0 for term 4 01:48:30,770 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:30,770 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 01:48:30,770 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 168.0 μs 01:48:30,770 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] and replicatedToAllIndex is 0 01:48:30,770 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=4, lastIndex=1, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:48:30,770 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 7 01:48:30,770 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:48:30,770 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:30,770 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:48:30,770 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=4, lastAppliedIndex=1, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5106465, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:48:30,770 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 1 01:48:30,770 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 4 01:48:30,770 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] 01:48:30,771 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 7 01:48:30,771 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 7 01:48:30,771 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 01:48:30,771 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 01:48:30,771 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a journal response 01:48:30,771 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 1, nextIndex: 2 01:48:30,771 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:48:30,771 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 1 01:48:30,771 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 01:48:30,771 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 1 data Simple{size=282} 01:48:30,771 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,772 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:48:30,772 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:48:30,772 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-199980842$b] 01:48:30,772 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: commit succeeded 01:48:30,772 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-728200507] with name member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher 01:48:30,772 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-728200507] 01:48:30,772 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:30,772 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:48:30,772 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 0 for term 4 01:48:30,772 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:30,772 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 01:48:30,772 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 128.6 μs 01:48:30,772 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] and replicatedToAllIndex is 0 01:48:30,772 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=4, lastIndex=1, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:48:30,772 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 7 01:48:30,772 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-728200507] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:30,772 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:48:30,772 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 01:48:30,773 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:30,773 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:30,773 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:30,773 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027)] for shard default, trying to use DirectTransactionContextWrapper 01:48:30,773 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 - Creating local component for shard default using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@59b9c916 01:48:30,773 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:30,776 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-525977620] 01:48:30,776 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 01:48:30,776 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:48:30,776 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=4, lastAppliedIndex=1, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@ac6f067, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:48:30,776 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 1 01:48:30,776 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 4 01:48:30,776 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 1 01:48:30,776 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:48:30,776 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 1 01:48:30,776 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 01:48:30,776 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 1 data Simple{size=113} 01:48:30,776 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,776 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-default-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-1023240453] with name member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher 01:48:30,776 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-1023240453] 01:48:30,777 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:30,777 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:48:30,777 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 0 for term 4 01:48:30,777 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 01:48:30,777 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 20.88 μs 01:48:30,777 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-1023240453] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:30,777 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] and replicatedToAllIndex is 0 01:48:30,777 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=4, lastIndex=1, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:48:30,777 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 01:48:30,777 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:48:30,778 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:48:30,778 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=4, lastAppliedIndex=1, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7d342cde, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:48:30,778 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 1 01:48:30,778 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 4 01:48:30,778 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 1 01:48:30,778 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:48:30,778 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 1 01:48:30,778 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 01:48:30,778 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 1 data Simple{size=282} 01:48:30,778 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:48:30,778 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-2045357147] with name member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher 01:48:30,778 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-2045357147] 01:48:30,779 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:30,779 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:48:30,779 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 0 for term 4 01:48:30,779 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842] 01:48:30,779 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 01:48:30,779 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 76.90 μs 01:48:30,779 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] and replicatedToAllIndex is 0 01:48:30,779 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=4, lastIndex=1, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:48:30,779 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-2045357147] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:30,779 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 01:48:30,779 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:48:30,779 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 01:48:30,779 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 01:48:30,779 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 01:48:30,779 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:48:30,779 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=4, lastAppliedIndex=1, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7be70388, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:48:30,779 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 1 01:48:30,779 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 4 01:48:30,779 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a journal response 01:48:30,779 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 01:48:30,779 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 01:48:30,779 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 01:48:30,779 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 7 01:48:30,779 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 7 01:48:30,780 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 01:48:30,780 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 01:48:30,780 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 01:48:30,780 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 01:48:30,780 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 01:48:30,780 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842] 01:48:30,780 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 01:48:30,780 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1897809026] 01:48:30,780 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 01:48:30,785 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-47345733] 01:48:30,785 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-47345733] 01:48:30,992 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 01:48:30,992 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:30,992 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:30,992 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:30,992 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 Readying 1 components for commit 01:48:30,992 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 Readying transaction for shard cars 01:48:30,992 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 = [ 1 ] 01:48:30,992 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:30,993 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 canCommit 01:48:30,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 01:48:30,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 01:48:30,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 01:48:30,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 validated 01:48:30,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:30,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: success 01:48:30,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 01:48:30,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 01:48:30,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 candidate ready 01:48:30,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: canCommit - candidate rootPath: / 01:48:30,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: canCommit - messages: [] 01:48:30,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 01:48:30,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 prepared 01:48:30,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:30,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 01:48:30,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 01:48:30,994 PM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:48:30,994 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}] 01:48:30,994 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}] 01:48:30,994 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:30,994 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:48:30,994 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:30,994 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}]]] 01:48:30,994 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 submitted to persistence 01:48:30,996 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}] 01:48:30,997 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}]]] 01:48:30,997 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:48:30,997 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:48:30,997 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 01:48:30,997 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}] 01:48:30,997 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 1 01:48:30,997 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:30,998 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}] 01:48:31,0 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 2, nextIndex: 3 01:48:31,0 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:48:31,0 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 2 01:48:31,0 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 1 to 2 01:48:31,0 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 2 data Simple{size=222} 01:48:31,0 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 01:48:31,0 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: commit - successfulFromPrevious: [] 01:48:31,0 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 01:48:31,0 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-199980842$c] 01:48:31,0 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 successfully completed direct commit 01:48:31,1 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 committed successfully 01:48:31,1 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,1 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:48:31,1 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=2 01:48:31,1 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-3-0 01:48:31,1 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 189.5 μs 01:48:31,1 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-3-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,1 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-3-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,1 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-3-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,1 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 01:48:31,1 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,1 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,1 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,1 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 Readying 1 components for commit 01:48:31,1 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 Readying transaction for shard cars 01:48:31,1 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 = [ 1 ] 01:48:31,1 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:31,1 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 canCommit 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 validated 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: success 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 candidate ready 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: canCommit - candidate rootPath: / 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: canCommit - messages: [] 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: preCommit - successfulFromPrevious: [] 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 prepared 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}] 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}] 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}]]] 01:48:31,2 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 submitted to persistence 01:48:31,3 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 2 01:48:31,3 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:48:31,3 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 2 01:48:31,3 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 1 to 2 01:48:31,3 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 2 data Simple{size=222} 01:48:31,3 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 01:48:31,3 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,4 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=2 01:48:31,4 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 68.16 μs 01:48:31,4 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}]]] 01:48:31,4 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:48:31,4 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:48:31,4 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 01:48:31,4 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}] 01:48:31,4 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:48:31,4 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:31,4 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}] 01:48:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 3, nextIndex: 4 01:48:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 01:48:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 3 01:48:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 2 to 3 01:48:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 3 data Simple{size=222} 01:48:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 01:48:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: commit - successfulFromPrevious: [] 01:48:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 changing state from COMMIT_PENDING to COMMITTED 01:48:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-199980842$d] 01:48:31,5 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 successfully completed direct commit 01:48:31,5 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 committed successfully 01:48:31,5 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 2 for term 4 01:48:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:48:31,5 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 44.92 μs 01:48:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=3 01:48:31,5 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}] 01:48:31,5 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-5-0 01:48:31,5 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-5-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,5 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-5-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,5 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-5-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,6 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 01:48:31,6 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,6 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,6 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,6 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 Readying 1 components for commit 01:48:31,6 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 Readying transaction for shard cars 01:48:31,6 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 = [ 1 ] 01:48:31,6 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:31,6 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 canCommit 01:48:31,6 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 01:48:31,6 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 01:48:31,6 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 01:48:31,6 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 validated 01:48:31,6 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:31,6 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: success 01:48:31,6 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 01:48:31,6 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 01:48:31,6 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 candidate ready 01:48:31,6 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: canCommit - candidate rootPath: / 01:48:31,6 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: canCommit - messages: [] 01:48:31,6 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: preCommit - successfulFromPrevious: [] 01:48:31,6 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 prepared 01:48:31,6 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:31,6 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 01:48:31,6 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 01:48:31,6 PM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:48:31,6 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}] 01:48:31,6 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}] 01:48:31,7 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:31,7 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 2 01:48:31,7 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:31,7 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}]]] 01:48:31,7 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 submitted to persistence 01:48:31,7 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}] 01:48:31,8 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 3 01:48:31,8 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 01:48:31,8 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 3 01:48:31,8 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 2 to 3 01:48:31,8 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 3 data Simple{size=222} 01:48:31,8 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 01:48:31,8 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,8 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 2 for term 4 01:48:31,8 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=3 01:48:31,8 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 70.95 μs 01:48:31,9 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:31,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}]]] 01:48:31,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:48:31,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 01:48:31,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 01:48:31,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}] 01:48:31,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:48:31,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:31,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}] 01:48:31,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 4, nextIndex: 5 01:48:31,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 01:48:31,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 4 01:48:31,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 3 to 4 01:48:31,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 4 data Simple{size=222} 01:48:31,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 01:48:31,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: commit - successfulFromPrevious: [] 01:48:31,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 changing state from COMMIT_PENDING to COMMITTED 01:48:31,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-199980842$e] 01:48:31,10 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 successfully completed direct commit 01:48:31,10 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 committed successfully 01:48:31,10 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 3 for term 4 01:48:31,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:48:31,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=4 01:48:31,10 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 69.85 μs 01:48:31,10 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-7-0 01:48:31,10 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-7-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,10 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-7-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,10 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-7-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,11 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 01:48:31,11 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,11 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,11 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,11 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 Readying 1 components for commit 01:48:31,11 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 Readying transaction for shard cars 01:48:31,11 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 = [ 1 ] 01:48:31,11 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:31,11 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 canCommit 01:48:31,11 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 01:48:31,11 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 01:48:31,11 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 01:48:31,11 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 validated 01:48:31,11 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:31,11 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: success 01:48:31,11 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 01:48:31,11 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 01:48:31,11 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 candidate ready 01:48:31,11 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: canCommit - candidate rootPath: / 01:48:31,11 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: canCommit - messages: [] 01:48:31,11 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: preCommit - successfulFromPrevious: [] 01:48:31,11 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 prepared 01:48:31,12 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:31,12 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 01:48:31,12 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 01:48:31,12 PM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:48:31,12 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 01:48:31,12 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 01:48:31,12 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:31,12 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 3 01:48:31,12 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:31,12 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}]]] 01:48:31,12 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 submitted to persistence 01:48:31,13 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 01:48:31,13 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 4 01:48:31,13 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4 01:48:31,13 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 4 01:48:31,13 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 3 to 4 01:48:31,13 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 4 data Simple{size=222} 01:48:31,13 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 01:48:31,14 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,14 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 83.21 μs 01:48:31,14 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 3 for term 4 01:48:31,14 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}]]] 01:48:31,14 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:48:31,14 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 01:48:31,14 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 01:48:31,14 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 01:48:31,14 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:48:31,14 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=5, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:31,15 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:31,15 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 5, nextIndex: 6 01:48:31,15 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 01:48:31,15 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 5 01:48:31,15 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 01:48:31,15 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 4 to 5 01:48:31,15 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 5 data Simple{size=222} 01:48:31,15 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 01:48:31,15 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: commit - successfulFromPrevious: [] 01:48:31,15 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=4 01:48:31,15 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 changing state from COMMIT_PENDING to COMMITTED 01:48:31,16 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-199980842$f] 01:48:31,16 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 4 for term 4 01:48:31,16 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 01:48:31,16 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 successfully completed direct commit 01:48:31,16 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 committed successfully 01:48:31,16 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,16 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-9-0 01:48:31,16 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-9-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,16 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-9-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,16 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-9-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,17 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 01:48:31,17 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,17 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,17 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,17 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 Readying 1 components for commit 01:48:31,17 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 Readying transaction for shard cars 01:48:31,17 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 = [ 1 ] 01:48:31,17 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:31,17 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 canCommit 01:48:31,17 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 69.16 μs 01:48:31,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 5 01:48:31,17 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=5 01:48:31,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5 01:48:31,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 5 01:48:31,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 4 to 5 01:48:31,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 5 data Simple{size=222} 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 validated 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: success 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 01:48:31,18 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 candidate ready 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: canCommit - candidate rootPath: / 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: canCommit - messages: [] 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: preCommit - successfulFromPrevious: [] 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 prepared 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}] 01:48:31,18 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}] 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 6, leaderLastIndex: 6, leaderSnapShotIndex: 4 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 6 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:31,18 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 75.13 μs 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}]]] 01:48:31,18 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 4 for term 4 01:48:31,18 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 submitted to persistence 01:48:31,18 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=5 01:48:31,19 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 6 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:31,19 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}] 01:48:31,21 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}]]] 01:48:31,21 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:48:31,21 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 5, entries to be added from: 0 01:48:31,21 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 01:48:31,21 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}] 01:48:31,21 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:48:31,21 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:31,21 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 6, nextIndex: 7 01:48:31,21 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 5 01:48:31,21 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}] 01:48:31,21 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 6 01:48:31,21 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 5 to 6 01:48:31,21 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 6 data Simple{size=222} 01:48:31,21 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 01:48:31,21 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: commit - successfulFromPrevious: [] 01:48:31,21 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 changing state from COMMIT_PENDING to COMMITTED 01:48:31,22 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-199980842$g] 01:48:31,22 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 successfully completed direct commit 01:48:31,22 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 committed successfully 01:48:31,22 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,22 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-11-0 01:48:31,22 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-11-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,22 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-11-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,22 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-11-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,22 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 64.35 μs 01:48:31,22 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 5 for term 4 01:48:31,22 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 01:48:31,22 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 01:48:31,22 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,22 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,22 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,22 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=6 01:48:31,22 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 Readying 1 components for commit 01:48:31,22 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 Readying transaction for shard cars 01:48:31,22 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 = [ 1 ] 01:48:31,22 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:31,22 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 canCommit 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 validated 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: success 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 candidate ready 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: canCommit - candidate rootPath: / 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: canCommit - messages: [] 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: preCommit - successfulFromPrevious: [] 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 prepared 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}] 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}] 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: 5 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 7 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}]]] 01:48:31,23 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 submitted to persistence 01:48:31,24 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}] 01:48:31,24 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 6 01:48:31,24 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 5, lastIndex(): 6 01:48:31,24 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 6 01:48:31,24 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 5 to 6 01:48:31,24 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 6 data Simple{size=222} 01:48:31,25 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 01:48:31,25 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,25 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 5 for term 4 01:48:31,25 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}]]] 01:48:31,25 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 73.55 μs 01:48:31,25 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:48:31,25 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 01:48:31,25 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 01:48:31,25 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}] 01:48:31,25 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:48:31,25 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=7, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:31,26 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}] 01:48:31,26 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 7 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:31,26 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=6 01:48:31,26 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 7, nextIndex: 8 01:48:31,26 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 6 01:48:31,26 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 7 01:48:31,26 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 6 to 7 01:48:31,26 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 7 data Simple{size=222} 01:48:31,26 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 01:48:31,26 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: commit - successfulFromPrevious: [] 01:48:31,26 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 changing state from COMMIT_PENDING to COMMITTED 01:48:31,26 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-199980842$h] 01:48:31,26 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 6 for term 4 01:48:31,26 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 01:48:31,27 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=7 01:48:31,27 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 committed successfully 01:48:31,27 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 successfully completed direct commit 01:48:31,27 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,27 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 57.46 μs 01:48:31,27 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-13-0 01:48:31,27 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-13-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,27 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-13-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,27 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-13-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,27 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 01:48:31,27 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,27 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,27 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,27 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 Readying 1 components for commit 01:48:31,27 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 Readying transaction for shard cars 01:48:31,27 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 = [ 1 ] 01:48:31,27 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:31,27 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 canCommit 01:48:31,27 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 01:48:31,27 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 01:48:31,27 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 01:48:31,27 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 7 01:48:31,27 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 validated 01:48:31,27 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:31,27 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: success 01:48:31,27 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 01:48:31,27 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 7, context.getLastApplied(): 6, lastIndex(): 7 01:48:31,28 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 7 01:48:31,28 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 6 to 7 01:48:31,28 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 7 data Simple{size=222} 01:48:31,27 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 01:48:31,28 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 candidate ready 01:48:31,28 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: canCommit - candidate rootPath: / 01:48:31,28 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 01:48:31,28 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: canCommit - messages: [] 01:48:31,28 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: preCommit - successfulFromPrevious: [] 01:48:31,28 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 prepared 01:48:31,28 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:31,28 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 01:48:31,28 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 01:48:31,29 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,29 PM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:48:31,29 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}] 01:48:31,29 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}] 01:48:31,29 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0, logIndex: 8, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:31,29 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 61.62 μs 01:48:31,29 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 8, leaderLastIndex: 8, leaderSnapShotIndex: 6 01:48:31,29 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 8 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:31,29 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 6 for term 4 01:48:31,29 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=7, prevLogTerm=4, leaderCommit=7, replicatedToAllIndex=6, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}]]] 01:48:31,29 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=7 01:48:31,29 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 submitted to persistence 01:48:31,29 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 8 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:31,29 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}] 01:48:31,32 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=7, prevLogTerm=4, leaderCommit=7, replicatedToAllIndex=6, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}]]] 01:48:31,32 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:48:31,32 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 7, entries to be added from: 0 01:48:31,32 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 01:48:31,32 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}] 01:48:31,32 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:48:31,32 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=8, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:31,33 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}] 01:48:31,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 8, nextIndex: 9 01:48:31,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 8, lastAppliedIndex: 7 01:48:31,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 8 01:48:31,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 7 to 8 01:48:31,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 8 data Simple{size=222} 01:48:31,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 01:48:31,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: commit - successfulFromPrevious: [] 01:48:31,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 changing state from COMMIT_PENDING to COMMITTED 01:48:31,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-199980842$i] 01:48:31,34 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 successfully completed direct commit 01:48:31,34 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 committed successfully 01:48:31,34 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,34 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-15-0 01:48:31,34 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-15-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,34 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-15-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,34 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 70.24 μs 01:48:31,34 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-15-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,34 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 7 for term 4 01:48:31,34 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 01:48:31,34 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: 7 01:48:31,34 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=8 01:48:31,34 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,34 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,34 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,34 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 Readying 1 components for commit 01:48:31,34 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 Readying transaction for shard cars 01:48:31,34 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 = [ 1 ] 01:48:31,34 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:31,34 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 canCommit 01:48:31,34 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 validated 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: success 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 candidate ready 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: canCommit - candidate rootPath: / 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: canCommit - messages: [] 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: preCommit - successfulFromPrevious: [] 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 prepared 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}] 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}] 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0, logIndex: 9, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 9, leaderLastIndex: 9, leaderSnapShotIndex: 7 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 9 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=8, prevLogTerm=4, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}]]] 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 submitted to persistence 01:48:31,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}] 01:48:31,40 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 8 01:48:31,40 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 8, context.getLastApplied(): 7, lastIndex(): 8 01:48:31,40 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 8 01:48:31,40 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 7 to 8 01:48:31,40 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 8 data Simple{size=222} 01:48:31,41 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 01:48:31,41 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,41 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 69.52 μs 01:48:31,41 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 7 for term 4 01:48:31,41 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=8, prevLogTerm=4, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}]]] 01:48:31,41 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:48:31,41 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 8, entries to be added from: 0 01:48:31,41 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 01:48:31,41 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}] 01:48:31,41 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:48:31,41 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=9, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:31,42 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 9 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:31,43 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}] 01:48:31,43 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=8 01:48:31,48 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 9, nextIndex: 10 01:48:31,48 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 9, lastAppliedIndex: 8 01:48:31,48 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 9 01:48:31,48 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 8 to 9 01:48:31,48 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 9 data Simple{size=222} 01:48:31,48 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 01:48:31,48 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: commit - successfulFromPrevious: [] 01:48:31,48 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 changing state from COMMIT_PENDING to COMMITTED 01:48:31,48 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-199980842$j] 01:48:31,48 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 8 for term 4 01:48:31,49 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 10, leaderLastIndex: 9, leaderSnapShotIndex: 8 01:48:31,49 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=9 01:48:31,51 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 committed successfully 01:48:31,51 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,51 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 successfully completed direct commit 01:48:31,51 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-17-0 01:48:31,51 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 86.70 μs 01:48:31,51 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-17-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,51 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-17-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,51 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-17-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,52 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 01:48:31,52 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,52 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,52 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,52 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 Readying 1 components for commit 01:48:31,52 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 Readying transaction for shard cars 01:48:31,52 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 = [ 1 ] 01:48:31,52 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:31,52 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 canCommit 01:48:31,52 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 01:48:31,52 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 01:48:31,52 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 01:48:31,52 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 validated 01:48:31,52 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:31,52 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: success 01:48:31,52 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 01:48:31,52 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 01:48:31,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 candidate ready 01:48:31,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: canCommit - candidate rootPath: / 01:48:31,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: canCommit - messages: [] 01:48:31,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: preCommit - successfulFromPrevious: [] 01:48:31,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 prepared 01:48:31,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:31,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 01:48:31,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 01:48:31,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:48:31,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}] 01:48:31,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}] 01:48:31,53 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 9 01:48:31,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0, logIndex: 10, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:31,53 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 9, context.getLastApplied(): 8, lastIndex(): 9 01:48:31,53 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 9 01:48:31,53 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 8 to 9 01:48:31,53 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 9 data Simple{size=222} 01:48:31,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 10, leaderLastIndex: 10, leaderSnapShotIndex: 8 01:48:31,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 10 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:31,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=9, prevLogTerm=4, leaderCommit=9, replicatedToAllIndex=8, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}]]] 01:48:31,53 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 submitted to persistence 01:48:31,53 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 01:48:31,54 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,54 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 81.19 μs 01:48:31,54 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 8 for term 4 01:48:31,54 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=9 01:48:31,59 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 10 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:31,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=9, prevLogTerm=4, leaderCommit=9, replicatedToAllIndex=8, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}]]] 01:48:31,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:48:31,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 9, entries to be added from: 0 01:48:31,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 01:48:31,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}] 01:48:31,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:48:31,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=10, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:31,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}] 01:48:31,61 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 10, nextIndex: 11 01:48:31,61 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 11, leaderLastIndex: 10, leaderSnapShotIndex: 8 01:48:31,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}] 01:48:31,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 10, lastAppliedIndex: 9 01:48:31,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 10 01:48:31,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 9 to 10 01:48:31,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 10 data Simple{size=222} 01:48:31,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 01:48:31,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: commit - successfulFromPrevious: [] 01:48:31,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 changing state from COMMIT_PENDING to COMMITTED 01:48:31,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-199980842$k] 01:48:31,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 9 for term 4 01:48:31,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=10 01:48:31,65 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 successfully completed direct commit 01:48:31,65 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 committed successfully 01:48:31,65 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,65 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-19-0 01:48:31,65 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-19-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,65 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-19-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,65 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-19-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,65 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 100.1 μs 01:48:31,65 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 01:48:31,65 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,65 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,65 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,65 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 Readying 1 components for commit 01:48:31,65 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 Readying transaction for shard cars 01:48:31,65 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 = [ 1 ] 01:48:31,65 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:31,66 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 canCommit 01:48:31,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 01:48:31,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 01:48:31,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 01:48:31,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 validated 01:48:31,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:31,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: success 01:48:31,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 01:48:31,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 01:48:31,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 candidate ready 01:48:31,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: canCommit - candidate rootPath: / 01:48:31,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: canCommit - messages: [] 01:48:31,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: preCommit - successfulFromPrevious: [] 01:48:31,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 prepared 01:48:31,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:31,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 01:48:31,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 01:48:31,68 PM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:48:31,68 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}] 01:48:31,68 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}] 01:48:31,68 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0, logIndex: 11, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:31,68 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 11, leaderLastIndex: 11, leaderSnapShotIndex: 9 01:48:31,68 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 11 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:31,68 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=10, prevLogTerm=4, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}]]] 01:48:31,68 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 submitted to persistence 01:48:31,69 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}] 01:48:31,69 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=10, prevLogTerm=4, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}]]] 01:48:31,69 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:48:31,69 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 10, entries to be added from: 0 01:48:31,69 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 01:48:31,69 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}] 01:48:31,69 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 3 01:48:31,69 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 10 01:48:31,69 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=11, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:31,69 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 10, context.getLastApplied(): 9, lastIndex(): 11 01:48:31,69 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 10 01:48:31,69 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 9 to 10 01:48:31,69 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 10 data Simple{size=222} 01:48:31,70 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 01:48:31,70 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 9 for term 4 01:48:31,71 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,71 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 96.53 μs 01:48:31,71 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 11, nextIndex: 12 01:48:31,71 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 11, lastAppliedIndex: 10 01:48:31,71 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 11 01:48:31,71 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 10 to 11 01:48:31,71 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 11 data Simple{size=222} 01:48:31,71 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 01:48:31,71 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: commit - successfulFromPrevious: [] 01:48:31,72 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 changing state from COMMIT_PENDING to COMMITTED 01:48:31,72 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-199980842$l] 01:48:31,72 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 successfully completed direct commit 01:48:31,72 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,72 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 committed successfully 01:48:31,72 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 75.42 μs 01:48:31,72 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 10 for term 4 01:48:31,72 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 12, leaderLastIndex: 11, leaderSnapShotIndex: 10 01:48:31,72 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=11 01:48:31,73 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-21-0 01:48:31,73 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-21-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,73 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-21-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,73 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-21-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,74 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-525977620] 01:48:31,74 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}] 01:48:31,74 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=10 01:48:31,74 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 01:48:31,74 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842] 01:48:31,74 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 11 01:48:31,74 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 11, context.getLastApplied(): 10, lastIndex(): 11 01:48:31,74 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 11 01:48:31,74 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 10 to 11 01:48:31,74 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 11 data Simple{size=222} 01:48:31,74 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 01:48:31,75 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,75 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 10 for term 4 01:48:31,75 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=11 01:48:31,75 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 110.1 μs 01:48:31,75 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842] 01:48:31,76 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1897809026] 01:48:31,76 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 01:48:31,76 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-47345733] 01:48:31,76 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-47345733] 01:48:31,279 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,279 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,279 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,279 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,279 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027)] for shard default, trying to use DirectTransactionContextWrapper 01:48:31,279 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 - Creating local component for shard default using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@59b9c916 01:48:31,279 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,279 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 Readying 2 components for commit 01:48:31,279 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 Readying transaction for shard cars 01:48:31,279 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 = [ 1 ] 01:48:31,279 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:31,280 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 Readying transaction for shard default 01:48:31,280 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 = [ 1 ] 01:48:31,280 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027)] 01:48:31,280 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 canCommit 01:48:31,280 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,280 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,280 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,280 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,280 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransactionReadyReplyMapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 readyTransaction 01:48:31,280 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] resolved 01:48:31,280 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 resolved to actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:31,280 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransactionReadyReplyMapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 readyTransaction 01:48:31,280 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027)] resolved 01:48:31,280 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: successfully resolved all cohort actors 01:48:31,280 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 finishCanCommit 01:48:31,280 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: sending CanCommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:31,281 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:31,281 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 resolved to actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027)] 01:48:31,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Can committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 validated 01:48:31,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:31,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: success 01:48:31,281 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:48:31,281 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: sending CanCommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027)] 01:48:31,281 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027)] 01:48:31,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Can committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 validated 01:48:31,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:31,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: success 01:48:31,282 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:48:31,282 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit returning result: true 01:48:31,282 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 commit 01:48:31,282 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 finish commit 01:48:31,282 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Sending CommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:31,282 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] 01:48:31,282 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Sending CommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027)] 01:48:31,282 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027)] 01:48:31,282 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,282 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,282 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 candidate ready 01:48:31,282 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit - candidate rootPath: / 01:48:31,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 candidate ready 01:48:31,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit - candidate rootPath: / 01:48:31,283 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit - messages: [] 01:48:31,283 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: preCommit - successfulFromPrevious: [] 01:48:31,283 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 prepared 01:48:31,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit - messages: [] 01:48:31,283 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:31,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: preCommit - successfulFromPrevious: [] 01:48:31,283 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 prepared 01:48:31,283 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:31,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 114 01:48:31,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] 01:48:31,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] 01:48:31,283 PM [cluster-test-shard-dispatcher-33] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 283 01:48:31,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:31,283 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] 01:48:31,283 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] 01:48:31,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:48:31,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 2 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 01:48:31,283 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, logIndex: 12, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:31,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}]]] 01:48:31,283 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 12, leaderLastIndex: 12, leaderSnapShotIndex: 10 01:48:31,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 submitted to persistence 01:48:31,283 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 12 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:31,283 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=11, prevLogTerm=4, leaderCommit=11, replicatedToAllIndex=10, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}]]] 01:48:31,284 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 submitted to persistence 01:48:31,284 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] 01:48:31,284 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] 01:48:31,285 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}]]] 01:48:31,285 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:48:31,285 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:48:31,285 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=114} 01:48:31,285 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] 01:48:31,285 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=11, prevLogTerm=4, leaderCommit=11, replicatedToAllIndex=10, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}]]] 01:48:31,285 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:48:31,285 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 11, entries to be added from: 0 01:48:31,285 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=283} 01:48:31,285 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] 01:48:31,285 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 1 01:48:31,285 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:31,285 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:48:31,285 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=12, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:31,286 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] 01:48:31,286 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] 01:48:31,286 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 2, nextIndex: 3 01:48:31,286 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:48:31,286 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 12, nextIndex: 13 01:48:31,286 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 2 01:48:31,286 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 1 to 2 01:48:31,286 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 2 data Simple{size=114} 01:48:31,286 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 12, lastAppliedIndex: 11 01:48:31,286 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,286 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 12 01:48:31,286 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 11 to 12 01:48:31,286 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: commit - successfulFromPrevious: [] 01:48:31,286 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 12 data Simple{size=283} 01:48:31,286 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from COMMIT_PENDING to COMMITTED 01:48:31,286 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,286 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-default-testSnapshotOnRootOverwrite%231227224027$m] 01:48:31,286 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: commit - successfulFromPrevious: [] 01:48:31,286 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from COMMIT_PENDING to COMMITTED 01:48:31,286 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-199980842$n] 01:48:31,286 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:48:31,286 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:48:31,287 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=2 01:48:31,287 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] and replicatedToAllIndex is 1 01:48:31,287 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=4, lastIndex=2, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:48:31,287 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 9 01:48:31,287 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:48:31,287 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:48:31,287 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 11 for term 4 01:48:31,287 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: commit succeeded 01:48:31,287 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,287 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 13, leaderLastIndex: 12, leaderSnapShotIndex: 11 01:48:31,287 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,287 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=12 01:48:31,287 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] and replicatedToAllIndex is 11 01:48:31,287 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 85.34 μs 01:48:31,287 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 97.65 μs 01:48:31,287 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=12, lastAppliedTerm=4, lastIndex=12, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:48:31,287 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 29 01:48:31,287 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0 01:48:31,287 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:48:31,287 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:31,287 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:48:31,287 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a76dc8e 01:48:31,287 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,288 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:48:31,288 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=2, lastTerm=4, lastAppliedIndex=2, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@761404d2, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:48:31,288 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1227224027)] for shard default, trying to use DirectTransactionContextWrapper 01:48:31,288 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 2 01:48:31,288 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0 - Creating local component for shard default using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@59b9c916 01:48:31,288 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:31,288 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 2 and term: 4 01:48:31,288 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=12, lastTerm=4, lastAppliedIndex=12, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@32f911b9, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:48:31,288 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 12 01:48:31,288 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 12 and term: 4 01:48:31,288 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-525977620] 01:48:31,289 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 01:48:31,289 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842] 01:48:31,289 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 9 01:48:31,289 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 9 01:48:31,289 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 01:48:31,289 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 29 01:48:31,289 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 29 01:48:31,289 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 01:48:31,289 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 01:48:31,289 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 01:48:31,289 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a journal response 01:48:31,289 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 01:48:31,289 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842] 01:48:31,289 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 2 01:48:31,289 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:48:31,289 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 2 01:48:31,289 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 1 to 2 01:48:31,289 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 2 data Simple{size=114} 01:48:31,290 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 12 01:48:31,290 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,290 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 12, context.getLastApplied(): 11, lastIndex(): 12 01:48:31,290 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 12 01:48:31,290 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 11 to 12 01:48:31,290 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 12 data Simple{size=283} 01:48:31,290 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:48:31,290 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=2 01:48:31,290 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:48:31,290 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] and replicatedToAllIndex is 1 01:48:31,290 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=4, lastIndex=2, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:48:31,290 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 7 01:48:31,290 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:48:31,290 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,290 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:48:31,290 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 11 for term 4 01:48:31,290 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 61.33 μs 01:48:31,290 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=12 01:48:31,290 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:48:31,290 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] and replicatedToAllIndex is 11 01:48:31,290 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=12, lastAppliedTerm=4, lastIndex=12, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:48:31,290 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 27 01:48:31,290 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 13.01 μs 01:48:31,290 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:48:31,290 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:48:31,290 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=2, lastTerm=4, lastAppliedIndex=2, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@3269155b, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:48:31,290 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 2 01:48:31,290 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 2 and term: 4 01:48:31,291 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:48:31,291 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=12, lastTerm=4, lastAppliedIndex=12, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@564a5335, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:48:31,291 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 12 01:48:31,291 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 12 and term: 4 01:48:31,291 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1897809026] 01:48:31,291 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 01:48:31,291 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-47345733] 01:48:31,292 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-47345733] 01:48:31,292 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 7 01:48:31,292 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 7 01:48:31,292 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 01:48:31,292 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 27 01:48:31,292 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 27 01:48:31,292 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 01:48:31,293 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 01:48:31,293 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a journal response 01:48:31,293 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 01:48:31,293 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 01:48:31,482 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:48:31,495 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:48:31,496 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-1-shard-cars-testSnapshotOnRootOverwrite 01:48:31,496 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-1-shard-default-testSnapshotOnRootOverwrite 01:48:31,496 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSnapshotOnRootOverwrite - waiting on 2 shards 01:48:31,496 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onShutDown 01:48:31,496 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating leader transfer 01:48:31,496 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onShutDown 01:48:31,496 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating leader transfer 01:48:31,496 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3217db14 01:48:31,496 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@69206599 01:48:31,496 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3217db14 01:48:31,496 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Attempting to transfer leadership 01:48:31,496 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@69206599 01:48:31,496 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Attempting to transfer leadership 01:48:31,496 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 2 01:48:31,496 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: false, followerNextIndex: 13, leaderLastIndex: 12, leaderSnapShotIndex: 12 01:48:31,496 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 01:48:31,497 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 01:48:31,497 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 01:48:31,497 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:31,497 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:31,497 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 01:48:31,501 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite] 01:48:31,501 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 01:48:31,501 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 01:48:31,501 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite] 01:48:31,502 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 01:48:31,502 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 01:48:31,502 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:31,502 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:31,503 PM [cluster-test-shard-dispatcher-33] [WARN] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=12, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 210, lastApplied : 12, commitIndex : 12 01:48:31,503 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSnapshotOnRootOverwrite, matchIndex: 12, lastIndex: 12, isVoting: true 01:48:31,503 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Follower's log matches - sending ElectionTimeout 01:48:31,503 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 13, leaderLastIndex: 12, leaderSnapShotIndex: 12 01:48:31,503 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-default-testSnapshotOnRootOverwrite, matchIndex: 2, lastIndex: 2, isVoting: true 01:48:31,503 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Follower's log matches - sending ElectionTimeout 01:48:31,503 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Leader transfer complete 01:48:31,503 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 2 01:48:31,504 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Leader transfer complete 01:48:31,911 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All shards shutdown - sending PoisonPill to self 01:48:31,911 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] ShardManager - shard-manager-testSnapshotOnRootOverwrite: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-199980842]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:48:31,911 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSnapshotOnRootOverwrite 01:48:31,911 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageSlicer - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader)_slicer-id-445: Closing 01:48:31,912 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 01:48:31,912 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - Stopping Shard member-1-shard-default-testSnapshotOnRootOverwrite 01:48:31,912 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageSlicer - member-1-shard-default-testSnapshotOnRootOverwrite (Leader)_slicer-id-444: Closing 01:48:31,912 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 01:48:31,912 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-testSnapshotOnRootOverwrite 01:48:31,912 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 01:48:31,912 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:48:31,913 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: shutting down 01:48:31,913 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 01:48:31,913 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 01:48:31,913 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-525977620] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-525977620] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:31,913 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:31,913 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:31,920 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:31,920 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:31,920 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:31,920 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:31,922 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:48:31,934 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSnapshotOnRootOverwrite: Closing 01:48:31,934 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSnapshotOnRootOverwrite 01:48:31,934 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Closing 01:48:31,934 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-default-testSnapshotOnRootOverwrite 01:48:31,934 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Closing 01:48:31,934 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 01:48:31,934 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 01:48:31,935 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-testSnapshotOnRootOverwrite 01:48:31,935 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:31,935 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:32,934 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [1038413523242514956] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:32,954 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-4382350408938990469] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:32,956 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:32,957 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:32,959 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:32.962] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:32,978 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:32,978 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/11/2024 13:48:32.983] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:33,3 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:48:33,9 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [1006061004735126973] 01:48:33,9 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:48:33,11 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:48:33,11 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:48:33,16 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:33,16 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:33,21 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:48:33,21 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:48:33,21 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:48:33,24 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:33,29 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [3495115231080752501] 01:48:33,29 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:48:33,31 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:33,31 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:48:33,31 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:33,31 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:33,48 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:48:33,75 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-9018947012037276695] 01:48:33,76 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:48:33,77 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:33,77 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:48:33,86 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:33,87 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:33,88 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 01:48:33,95 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:33,95 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:33,101 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:33,103 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-1137981597] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:33,103 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:48:33,103 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 01:48:33,105 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 01:48:33,105 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-1-shard-cars-testWriteTransactionWithSingleShard 01:48:33,105 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:33,105 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:48:33,106 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: no initial generation override, starting from 0 01:48:33,106 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:48:33,106 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@445b85b9 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2948bf7 01:48:33,106 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@479605c1 01:48:33,106 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1710164913106, None)) 01:48:33,106 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164913105,0,0)) 01:48:33,106 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4eaf8f60 01:48:33,106 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2948bf7 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5ce0ba12 01:48:33,106 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} started 01:48:33,107 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:33,108 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:33,110 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 01:48:33,111 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:33,111 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testWriteTransactionWithSingleShard 01:48:33,111 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:48:33,111 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:48:33,111 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard on shard member-1-shard-cars-testWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1730133323] 01:48:33,112 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-1-shard-cars-testWriteTransactionWithSingleShard, persistent : true 01:48:33,112 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-1-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 01:48:33,113 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#1964417335] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:33,114 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 01:48:33,114 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1730133323] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:33,114 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-member-1:snapshot-read#-1604048337] 01:48:33,114 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-notifier#1216012741 created and ready for shard:member-1-shard-cars-testWriteTransactionWithSingleShard 01:48:33,114 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-member-1:snapshot-read#-1604048337] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:33,117 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-notifier#1216012741] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:33,121 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:33,121 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:33,121 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:48:33,121 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 01:48:33,121 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 01:48:33,121 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-cars-testWriteTransactionWithSingleShard is initialized 01:48:33,122 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 01:48:33,122 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard 01:48:33,122 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from null to Follower 01:48:33,122 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: no initial generation override, starting from 0 01:48:33,122 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:48:33,122 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 01:48:33,122 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1bf2401e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4ce7f8cf 01:48:33,122 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:33,123 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@27762f90 01:48:33,123 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:33,123 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:48:33,123 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1710164913123, None)) 01:48:33,126 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164913122,0,0)) 01:48:33,127 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@269111b8 01:48:33,127 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4ce7f8cf to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2cc80780 01:48:33,127 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} started 01:48:33,127 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:33,127 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:33,128 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:48:33,128 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:33,128 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard on shard member-2-shard-cars-testWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1184207929] 01:48:33,128 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithSingleShard, persistent : true 01:48:33,128 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1184207929] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:33,128 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-154045350 created and ready for shard:member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:33,128 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#286889089] 01:48:33,128 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-154045350] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:33,129 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#286889089] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:33,129 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 01:48:33,130 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:33,130 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:33,130 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:48:33,130 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 01:48:33,130 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 01:48:33,130 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:48:33,130 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 01:48:33,130 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithSingleShard is initialized 01:48:33,130 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard 01:48:33,131 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testWriteTransactionWithSingleShard from null to Follower 01:48:33,136 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:48:33,139 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:33,177 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-1137981597] 01:48:33,177 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 01:48:33,178 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1730133323] 01:48:33,369 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:48:33,370 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 01:48:33,370 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testWriteTransactionWithSingleShard] 01:48:33,370 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:48:33,370 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 1 01:48:33,370 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:33,370 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 01:48:33,370 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:48:33,370 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 01:48:33,370 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 01:48:33,371 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from Follower to Candidate 01:48:33,649 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:48:33,649 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:48:33,650 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 2 01:48:33,732 PM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:48:33,890 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:48:33,890 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:48:33,890 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 3 01:48:34,31 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:48:34,31 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:48:34,34 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:34,34 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard 01:48:34,34 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:48:34,34 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:34,34 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:34,34 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:48:34,34 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-cars-testWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:34,38 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:34,38 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard 01:48:34,38 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:34,38 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:34,38 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:34,38 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:34,95 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:48:34,160 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:48:34,160 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:48:34,160 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 4 01:48:34,160 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:34,162 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:34,162 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Cluster state: [] 01:48:34,162 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:48:34,162 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:48:34,162 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 01:48:34,162 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:48:34,162 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:48:34,162 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:48:34,165 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:48:34,165 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:48:34,165 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testWriteTransactionWithSingleShard] 01:48:34,165 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:34,165 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:48:34,165 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:48:34,165 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: became leader with frontend state for [] 01:48:34,167 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard: Closing 01:48:34,167 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testWriteTransactionWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:48:34,167 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithSingleShard status sync done false 01:48:34,167 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:48:34,167 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:34,167 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:34,167 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 01:48:34,168 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: -1, nextIndex: 0 01:48:34,168 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:34,169 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 01:48:34,169 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:48:34,169 PM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:34,169 PM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:34,169 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from Candidate to Leader 01:48:34,169 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 01:48:34,187 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testWriteTransactionWithSingleShard commit rateLimit adjusted to 40.88563184859886 01:48:34,187 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:34,187 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#1964417335] 01:48:34,187 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:48:34,187 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:48:34,187 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:48:34,187 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard, primaryVersion=13] 01:48:34,187 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard, primaryVersion=13] 01:48:34,187 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:48:34,187 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:48:34,188 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Readying 1 components for commit 01:48:34,188 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Readying transaction for shard cars 01:48:34,188 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:48:34,188 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars 01:48:34,188 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 canCommit 01:48:34,188 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - creating WRITE_ONLY transaction context 01:48:34,188 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:48:34,188 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:48:34,188 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 01:48:34,188 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 01:48:34,188 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 directCommit called 01:48:34,188 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 sending 4 batched modifications, ready: true 01:48:34,188 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 01:48:34,191 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, disabling transaction tracking 01:48:34,191 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 01:48:34,191 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 01:48:34,191 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:48:34,191 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:34,191 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:34,192 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}]]] 01:48:34,192 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying 4 batched modifications for Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:34,192 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Readying Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 of 4 operations, client version 13 01:48:34,192 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: readying transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:34,192 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 ready 01:48:34,192 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:34,192 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 validated 01:48:34,192 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:34,192 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: canCommit for member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: success 01:48:34,192 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Committing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:34,192 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:34,192 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 candidate ready 01:48:34,192 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:48:34,192 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: canCommit - messages: [] 01:48:34,192 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:48:34,192 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 prepared 01:48:34,192 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:34,192 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Finishing commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:34,193 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:34,193 PM [cluster-test-shard-dispatcher-15] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 245 01:48:34,193 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 01:48:34,193 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 01:48:34,193 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:34,193 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:34,193 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:34,193 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 submitted to persistence 01:48:34,193 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}]]] 01:48:34,193 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 01:48:34,193 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:48:34,193 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62} 01:48:34,193 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 01:48:34,193 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 1 01:48:34,193 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:34,194 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 01:48:34,194 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 01:48:34,194 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithSingleShard status sync done true 01:48:34,194 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 01:48:34,195 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 0, nextIndex: 1 01:48:34,195 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:48:34,195 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 0 01:48:34,195 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 01:48:34,195 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62} 01:48:34,195 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:48:34,195 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendMetadata - member-1-shard-cars-testWriteTransactionWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} does not match any client, pre-disabling client. 01:48:34,195 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, nanosAgo=3407065} is already disabled 01:48:34,195 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:34,195 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:34,195 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}]]] 01:48:34,195 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 01:48:34,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}]]] 01:48:34,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 01:48:34,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:48:34,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=245} 01:48:34,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 01:48:34,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 2 01:48:34,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 0 01:48:34,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:34,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:48:34,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 0 01:48:34,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 01:48:34,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62} 01:48:34,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:48:34,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} does not match any client, pre-disabling client. 01:48:34,196 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 1, nextIndex: 2 01:48:34,196 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:48:34,196 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 1 01:48:34,196 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 01:48:34,196 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 1 data Simple{size=245} 01:48:34,196 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:34,196 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:48:34,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 01:48:34,196 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:48:34,197 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 01:48:34,197 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testWriteTransactionWithSingleShard_member-1-shard-cars-testWriteTransactionWithSingleShard$a] 01:48:34,197 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-1770854966] with name member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher 01:48:34,197 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-1770854966] 01:48:34,197 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 01:48:34,197 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:48:34,197 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:34,197 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 01:48:34,197 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 140.3 μs 01:48:34,198 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-1770854966] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:34,199 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 modifications completed with CommitTransactionReply [version=13] 01:48:34,199 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 successfully completed direct commit 01:48:34,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 1 01:48:34,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:48:34,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 1 01:48:34,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 01:48:34,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 1 data Simple{size=245} 01:48:34,199 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 01:48:34,199 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:34,199 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:48:34,199 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - trying create transaction 01:48:34,199 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 01:48:34,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:34,199 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 Queuing TransactionOperation 01:48:34,200 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-1455633008] with name member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher 01:48:34,200 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-1455633008] 01:48:34,200 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 01:48:34,200 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:48:34,200 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 01:48:34,200 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-1455633008] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:34,200 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 119.1 μs 01:48:34,201 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 01:48:34,202 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_10#-895769482, transactionId=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0, version=13] 01:48:34,202 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:48:34,202 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_10#-895769482)] 01:48:34,204 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_10#-895769482] 01:48:34,204 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_10#-895769482] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:34,205 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 ReadData operation succeeded 01:48:34,205 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:34,205 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-1137981597] 01:48:34,205 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:48:34,206 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:48:34,206 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1730133323, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@244e15a0] 01:48:34,206 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1730133323, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@244e15a0] 01:48:34,206 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1730133323)] for shard cars 01:48:34,206 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 01:48:34,206 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4bf2cff4 01:48:34,206 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:48:34,206 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:34,206 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:48:34,206 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - creating WRITE_ONLY transaction context 01:48:34,206 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 executeDelete called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 01:48:34,206 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 Readying 1 components for commit 01:48:34,206 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 Readying transaction for shard cars 01:48:34,206 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 directCommit called 01:48:34,206 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 sending 1 batched modifications, ready: true 01:48:34,206 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 01:48:34,206 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 canCommit 01:48:34,207 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying 1 batched modifications for Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:48:34,207 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Readying Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 of 1 operations, client version 13 01:48:34,207 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: readying transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:48:34,207 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 ready 01:48:34,207 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:48:34,207 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 validated 01:48:34,207 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:34,207 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: canCommit for member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: success 01:48:34,207 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Committing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:48:34,207 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:48:34,207 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 candidate ready 01:48:34,207 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: canCommit - candidate rootPath: / 01:48:34,208 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: canCommit - messages: [] 01:48:34,208 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 01:48:34,208 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 prepared 01:48:34,208 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:34,208 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Finishing commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:48:34,208 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:48:34,208 PM [cluster-test-shard-dispatcher-15] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 212 01:48:34,208 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 01:48:34,208 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 01:48:34,208 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:34,208 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:48:34,208 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:34,208 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}]]] 01:48:34,208 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 submitted to persistence 01:48:34,209 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 01:48:34,209 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}]]] 01:48:34,209 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 01:48:34,209 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:48:34,209 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=212} 01:48:34,209 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 01:48:34,209 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 2 01:48:34,209 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:34,210 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 01:48:34,210 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 2, nextIndex: 3 01:48:34,210 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:48:34,210 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 2 01:48:34,210 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 01:48:34,210 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 2 data Simple{size=212} 01:48:34,210 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:48:34,210 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: commit - successfulFromPrevious: [] 01:48:34,210 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 01:48:34,210 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testWriteTransactionWithSingleShard_member-1-shard-cars-testWriteTransactionWithSingleShard$d] 01:48:34,210 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:48:34,210 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 1 for term 4 01:48:34,210 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:48:34,210 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=2 01:48:34,210 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 101.4 μs 01:48:34,211 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 modifications completed with CommitTransactionReply [version=13] 01:48:34,211 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 successfully completed direct commit 01:48:34,211 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 01:48:34,211 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:34,211 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:48:34,211 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - trying create transaction 01:48:34,211 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 01:48:34,211 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 Queuing TransactionOperation 01:48:34,211 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 2 01:48:34,212 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:48:34,212 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 2 01:48:34,212 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 01:48:34,212 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 2 data Simple{size=212} 01:48:34,212 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:48:34,212 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 1 for term 4 01:48:34,212 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=2 01:48:34,212 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:48:34,213 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 76.65 μs 01:48:34,213 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 01:48:34,213 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_11#-1639406255] 01:48:34,213 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_11#-1639406255] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:34,214 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_11#-1639406255, transactionId=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0, version=13] 01:48:34,214 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 executeRead DataExists called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 01:48:34,214 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.DataExists to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_11#-1639406255)] 01:48:34,215 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 DataExists operation succeeded 01:48:34,215 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 01:48:34,215 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:34,215 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:48:34,215 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - trying create transaction 01:48:34,215 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 01:48:34,215 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 Queuing TransactionOperation 01:48:34,216 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 01:48:34,216 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_12#-99995868] 01:48:34,216 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_12#-99995868] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:34,217 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_12#-99995868, transactionId=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0, version=13] 01:48:34,217 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:48:34,217 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_12#-99995868)] 01:48:34,218 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 ReadData operation succeeded 01:48:34,219 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 01:48:34,219 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1730133323)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:34,219 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4bf2cff4 01:48:34,219 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 01:48:34,219 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-1137981597] 01:48:34,219 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 01:48:34,219 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1730133323] 01:48:34,219 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1730133323] 01:48:34,271 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:48:34,271 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testWriteTransactionWithSingleShard 01:48:34,271 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader)_slicer-id-450: Closing 01:48:34,271 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:48:34,271 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 01:48:34,272 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:34,272 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:34,278 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:34,278 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:34,278 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:34,278 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:34,279 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:48:34,290 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:48:34,291 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:34,291 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 01:48:34,291 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:48:34,291 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 01:48:34,292 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:34,292 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:35,294 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [1006061004735126973] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:35,305 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [3495115231080752501] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:35,313 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:35,313 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:35,315 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:35.316] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:35,332 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:48:35,341 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://reinstated-member2@127.0.0.1:2559] with UID [4974022498173480827] 01:48:35,341 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:48:35,342 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Registered cluster JMX MBean [akka:type=Cluster] 01:48:35,342 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Started up successfully 01:48:35,343 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:35,343 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:35,352 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:35,354 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:35,355 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 01:48:35,361 PM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 01:48:35,361 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2108694120] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:35,363 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: recovered identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:48:35,363 PM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 01:48:35,363 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 01:48:35,363 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:35,363 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@134a4997 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@310531b1 01:48:35,363 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@312b6b56 01:48:35,363 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-people-testWriteTransactionWithSingleShard 01:48:35,363 PM [reinstated-member2-akka.actor.default-dispatcher-16] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1710164915363, None)) 01:48:35,364 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164915362,0,0)) 01:48:35,364 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6d31f41d 01:48:35,364 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@310531b1 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@512acefe 01:48:35,364 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} started 01:48:35,365 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:35,365 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:35,365 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testWriteTransactionWithSingleShard 01:48:35,365 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Shard created : member-2-shard-people-testWriteTransactionWithSingleShard, persistent : true 01:48:35,365 PM [reinstated-member2-shard-dispatcher-13] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithSingleShard, persistent : true 01:48:35,365 PM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#1725378631] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:35,365 PM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard#524330464] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:35,365 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard-notifier#1989269886 created and ready for shard:member-2-shard-people-testWriteTransactionWithSingleShard 01:48:35,365 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-51846004] 01:48:35,365 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-1815528471] 01:48:35,365 PM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard-notifier#1989269886] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:35,365 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-2-shard-people-testWriteTransactionWithSingleShard with journal batch size 1 01:48:35,365 PM [reinstated-member2-shard-dispatcher-13] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 01:48:35,366 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-886985622 created and ready for shard:member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:35,366 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://reinstated-member2/deadLetters] 01:48:35,366 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 01:48:35,366 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://reinstated-member2/deadLetters] 01:48:35,366 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:48:35,366 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 01:48:35,366 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 0, size: 62 01:48:35,366 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 01:48:35,366 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 1, size: 245 01:48:35,366 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 01:48:35,366 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 0 to 0 01:48:35,366 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 01:48:35,366 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: ignoring unhandled payload DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62} 01:48:35,367 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 01:48:35,367 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 01:48:35,367 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 1 to 1 01:48:35,367 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 01:48:35,367 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://reinstated-member2/deadLetters] 01:48:35,367 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:35,367 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@39167cbc}, purged=MutableUnsignedLongSet{size=0}} 01:48:35,367 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:48:35,367 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 01:48:35,367 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 01:48:35,367 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 2, size: 212 01:48:35,367 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 01:48:35,367 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 2 to 2 01:48:35,367 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 01:48:35,367 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:48:35,367 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 01:48:35,367 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://reinstated-member2/deadLetters] 01:48:35,368 PM [reinstated-member2-shard-dispatcher-12] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Recovery completed in 995.7 μs - Switching actor to Follower - last log index = 2, last log term = 4, snapshot index = -1, snapshot term = -1, journal size = 3 01:48:35,368 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:48:35,368 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 01:48:35,368 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 01:48:35,368 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Received TimeoutNow - switching to Candidate 01:48:35,368 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [] 01:48:35,368 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=5, votedFor=member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:35,368 PM [reinstated-member2-shard-dispatcher-12] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 5 01:48:35,368 PM [reinstated-member2-shard-dispatcher-12] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:48:35,368 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 01:48:35,368 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:48:35,368 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 01:48:35,368 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 01:48:35,368 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [people] 01:48:35,368 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-people-testWriteTransactionWithSingleShard is initialized 01:48:35,368 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 01:48:35,368 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithSingleShard is initialized 01:48:35,370 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:48:35,370 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [] 01:48:35,370 PM [reinstated-member2-shard-dispatcher-13] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:48:35,370 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:48:35,370 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: became leader with frontend state for [member-2-frontend-datastore-testWriteTransactionWithSingleShard] 01:48:35,370 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 01:48:35,370 PM [reinstated-member2-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-51846004] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:35,370 PM [reinstated-member2-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-1815528471] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:35,370 PM [reinstated-member2-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-886985622] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:35,370 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard 01:48:35,370 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:48:35,370 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithSingleShard is no longer the leader 01:48:35,370 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Follower): Received TimeoutNow - switching to Candidate 01:48:35,370 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [] 01:48:35,370 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-2-shard-people-testWriteTransactionWithSingleShard 01:48:35,370 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Starting new election term 1 01:48:35,370 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:35,370 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithSingleShard (Follower): Closing 01:48:35,370 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:48:35,370 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithSingleShard is no longer the leader 01:48:35,371 PM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:48:35,371 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:35,371 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:35,371 PM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testWriteTransactionWithSingleShard from Candidate to Leader 01:48:35,371 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from null to Follower 01:48:35,371 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 01:48:35,371 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:48:35,371 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [] 01:48:35,371 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 01:48:35,371 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:48:35,371 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: became leader with frontend state for [] 01:48:35,371 PM [reinstated-member2-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 01:48:35,371 PM [reinstated-member2-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , registered listener akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard 01:48:35,371 PM [reinstated-member2-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithSingleShard, leaderId=member-2-shard-people-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:48:35,372 PM [reinstated-member2-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:35,372 PM [reinstated-member2-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:35,372 PM [reinstated-member2-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-people-testWriteTransactionWithSingleShard from Candidate to Leader 01:48:35,372 PM [reinstated-member2-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 01:48:35,415 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2108694120] 01:48:35,415 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 01:48:35,415 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#1725378631] 01:48:35,415 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 01:48:35,415 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2108694120] 01:48:35,415 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:48:35,415 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 Queuing TransactionOperation 01:48:35,415 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#1725378631, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@651b7891] 01:48:35,415 PM [reinstated-member2-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#1725378631, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@651b7891] 01:48:35,416 PM [reinstated-member2-akka.actor.default-dispatcher-16] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0: Found primary ActorSelection[Anchor(akka://reinstated-member2/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#1725378631)] for shard cars 01:48:35,416 PM [reinstated-member2-akka.actor.default-dispatcher-16] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 01:48:35,416 PM [reinstated-member2-akka.actor.default-dispatcher-16] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@2622ab53 01:48:35,416 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 01:48:35,416 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:35,416 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-people-testWriteTransactionWithSingleShard 01:48:35,416 PM [reinstated-member2-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithSingleShard - waiting on 2 shards 01:48:35,416 PM [reinstated-member2-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onShutDown 01:48:35,416 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onShutDown 01:48:35,416 PM [reinstated-member2-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@18f91ce2 01:48:35,416 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@398322c6 01:48:35,416 PM [reinstated-member2-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@18f91ce2 01:48:35,416 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-people-testWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@398322c6 01:48:35,417 PM [reinstated-member2-shard-dispatcher-12] [INFO] TestShard - Stopping Shard member-2-shard-people-testWriteTransactionWithSingleShard 01:48:35,417 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] MessageSlicer - member-2-shard-people-testWriteTransactionWithSingleShard (Leader)_slicer-id-455: Closing 01:48:35,417 PM [reinstated-member2-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithSingleShard 01:48:35,417 PM [reinstated-member2-shard-dispatcher-21] [DEBUG] MessageSlicer - member-2-shard-cars-testWriteTransactionWithSingleShard (Leader)_slicer-id-454: Closing 01:48:35,417 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:48:35,417 PM [reinstated-member2-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:48:35,417 PM [reinstated-member2-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 01:48:35,417 PM [reinstated-member2-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 01:48:35,417 PM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:48:35,418 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:48:35,418 PM [reinstated-member2-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: shutting down 01:48:35,418 PM [reinstated-member2-akka.actor.default-dispatcher-6] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:48:35,418 PM [main] [WARN] ActorUtils - ShardManager for testWriteTransactionWithSingleShard data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:48:35,418 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:48:35,418 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://reinstated-member2/user/$a#1789026501] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/$a#1789026501] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:35,418 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2108694120] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2108694120] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:35,418 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2108694120] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2108694120] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:35,418 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:48:35,419 PM [main] [WARN] ActorUtils - ShardManager for testWriteTransactionWithSingleShard data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:48:35,419 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:48:35,419 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:35,419 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:35,421 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:48:35,446 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:35,446 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:35,448 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:35.449] [reinstated-member2-akka.actor.internal-dispatcher-5] [Cluster(akka://reinstated-member2)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:35,464 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:35,476 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-9134258060952718266] 01:48:35,476 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:48:35,478 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:48:35,478 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:48:35,485 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:35,485 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:35,487 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:48:35,487 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:48:35,487 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:48:35,490 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:35,495 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [925559536305288354] 01:48:35,495 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:48:35,497 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:35,497 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:48:35,507 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:35,507 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:35,520 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:35,535 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [488143571465797664] 01:48:35,535 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:48:35,540 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:35,540 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:48:35,542 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:35,542 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:35,547 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:35,548 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:35,549 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,568 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-403773237] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:35,568 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,569 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,570 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Creating local shard: member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,570 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:35,570 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,570 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: no initial generation override, starting from 0 01:48:35,570 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:48:35,570 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1131153 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@53b16bf8 01:48:35,570 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@52975e22 01:48:35,570 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, 0, 1710164915570, None)) 01:48:35,570 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164915569,0,0)) 01:48:35,571 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@53b16bf8 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6d024571 01:48:35,571 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} started 01:48:35,571 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:35,572 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:35,574 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,575 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7784ba8a 01:48:35,575 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:35,575 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,575 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,575 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,575 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: updated peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-109478991] 01:48:35,575 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, persistent : true 01:48:35,576 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-109478991] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:35,576 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#1205818879 created and ready for shard:member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,576 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with journal batch size 1 01:48:35,576 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#1205818879] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:35,576 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-1:snapshot-read#916060973] 01:48:35,579 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:35,579 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:35,579 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Initializing shard [cars] 01:48:35,579 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is initialized 01:48:35,579 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:48:35,579 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 01:48:35,579 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from null to Follower 01:48:35,579 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,579 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from null to Follower 01:48:35,582 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-1679895222] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:35,582 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-1:snapshot-read#916060973] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:35,585 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,589 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,589 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Creating local shard: member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,593 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:48:35,596 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:35,597 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:48:35,599 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: no initial generation override, starting from 0 01:48:35,599 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:48:35,599 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@65b7e823 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5452d4f 01:48:35,599 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@17ab93f 01:48:35,600 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, 0, 1710164915599, None)) 01:48:35,600 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164915598,0,0)) 01:48:35,600 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5452d4f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@46ecac38 01:48:35,600 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} started 01:48:35,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1b83e4eb 01:48:35,601 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:35,601 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,602 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, persistent : true 01:48:35,602 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-1271419439] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:35,603 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#1679341465 created and ready for shard:member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,603 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#1679341465] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:35,603 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with journal batch size 1 01:48:35,603 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-2:snapshot-read#-38128147] 01:48:35,603 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-2:snapshot-read#-38128147] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:35,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:35,604 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:35,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:48:35,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 01:48:35,604 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Initializing shard [cars] 01:48:35,604 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is initialized 01:48:35,604 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from null to Follower 01:48:35,604 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,604 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from null to Follower 01:48:35,606 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:35,607 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:35,607 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,607 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,607 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,651 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-403773237] 01:48:35,651 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: findLocalShard : cars 01:48:35,651 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-109478991] 01:48:35,871 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:48:35,871 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Received ElectionTimeout - switching to Candidate 01:48:35,871 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 01:48:35,871 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:35,871 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 1 01:48:35,871 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:35,871 PM [cluster-test-shard-dispatcher-16] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 01:48:35,871 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:48:35,871 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 01:48:35,872 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Follower to Candidate 01:48:35,872 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Follower to Candidate 01:48:36,161 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:48:36,161 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,161 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 2 01:48:36,188 PM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:48:36,421 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:48:36,421 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,421 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 3 01:48:36,502 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:48:36,503 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:48:36,503 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:36,504 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,504 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,504 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:36,504 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,504 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,504 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,510 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:36,510 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,510 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,510 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:36,510 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,510 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,560 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:48:36,651 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:48:36,651 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,651 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 4 01:48:36,651 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,654 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:36,654 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Cluster state: [] 01:48:36,654 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:48:36,654 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:48:36,654 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=null 01:48:36,654 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:48:36,654 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,654 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:48:36,656 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:48:36,656 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:48:36,656 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 01:48:36,656 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:36,656 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:48:36,656 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:48:36,657 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: became leader with frontend state for [] 01:48:36,657 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Candidate to Leader 01:48:36,657 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderPayloadVersion=13] 01:48:36,657 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:36,657 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:36,657 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Candidate to Leader 01:48:36,657 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All Shards are ready - data store testTransactionWithCreateTxFailureDueToNoLeader is ready 01:48:36,657 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 01:48:36,657 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-1679895222] 01:48:36,657 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 Queuing TransactionOperation 01:48:36,657 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:48:36,657 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 01:48:36,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:48:36,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,658 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader status sync done false 01:48:36,658 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderPayloadVersion=13] 01:48:36,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is ready - notifying 1 OnShardInitialized callbacks 01:48:36,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, primaryVersion=13] 01:48:36,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:36,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:36,658 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All Shards are ready - data store testTransactionWithCreateTxFailureDueToNoLeader is ready 01:48:36,658 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, primaryVersion=13] 01:48:36,659 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] for shard cars 01:48:36,659 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader updated: matchIndex: -1, nextIndex: 0 01:48:36,659 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 01:48:36,659 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 01:48:36,659 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:36,660 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, disabling transaction tracking 01:48:36,660 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 01:48:36,660 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 01:48:36,660 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:48:36,660 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:36,660 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,660 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}]]] 01:48:36,660 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Creating transaction : member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 01:48:36,660 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_13#1701975856] 01:48:36,661 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_13#1701975856] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:36,661 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 01:48:36,661 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}]]] 01:48:36,661 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Number of entries to be appended = 1 01:48:36,661 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:48:36,661 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74} 01:48:36,661 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 01:48:36,661 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Log size is now 1 01:48:36,661 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:36,661 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_13#1701975856, transactionId=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0, version=13] 01:48:36,662 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:48:36,662 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_13#1701975856)] 01:48:36,662 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 01:48:36,662 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader status sync done true 01:48:36,662 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader updated: matchIndex: 0, nextIndex: 1 01:48:36,662 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:48:36,662 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Setting last applied to 0 01:48:36,662 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Moving last applied index from -1 to 0 01:48:36,662 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74} 01:48:36,662 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:48:36,662 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} does not match any client, pre-disabling client. 01:48:36,662 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, nanosAgo=2773606} is already disabled 01:48:36,663 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:36,663 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Persisting ApplyJournalEntries with index=0 01:48:36,663 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Commit index set to 0 01:48:36,663 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:48:36,663 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Setting last applied to 0 01:48:36,663 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Moving last applied index from -1 to 0 01:48:36,663 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74} 01:48:36,663 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:48:36,663 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} does not match any client, pre-disabling client. 01:48:36,663 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Persisting ApplyJournalEntries with index=0 01:48:36,664 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 ReadData operation succeeded 01:48:36,666 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:48:36,666 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,666 PM [cluster-test-shard-dispatcher-14] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader)_slicer-id-460: Closing 01:48:36,666 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Aborting 0 pending queued transactions 01:48:36,666 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,666 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:36,666 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:36,675 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:36,676 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:36,676 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:36,676 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:36,678 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:48:36,695 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 01:48:36,794 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,794 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,794 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 01:48:36,794 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionWithCreateTxFailureDueToNoLeader commit rateLimit adjusted to 40.88563184859886 01:48:36,794 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,794 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 01:48:36,794 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 01:48:36,794 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : true 01:48:36,794 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 01:48:36,795 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:36,795 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:48:36,795 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 01:48:36,795 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 01:48:36,795 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Queuing TransactionOperation 01:48:36,795 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Readying 1 components for commit 01:48:36,795 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Readying transaction for shard cars 01:48:36,795 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Queuing TransactionOperation 01:48:36,795 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 canCommit 01:48:36,818 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)]] after [10 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 01:48:36,848 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 01:48:36,850 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 01:48:36,850 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 01:48:36,879 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)]] after [10 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 01:48:36,908 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 01:48:36,908 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 01:48:36,908 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 01:48:36,938 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)]] after [10 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 01:48:36,968 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 01:48:36,968 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 01:48:36,968 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 01:48:36,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 01:48:36,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 01:48:36,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 01:48:36,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Received ElectionTimeout - switching to Candidate 01:48:36,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 01:48:36,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,979 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 5 01:48:36,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:36,979 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:48:36,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 01:48:36,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:48:36,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 01:48:36,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:48:36,980 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Follower to Candidate 01:48:36,980 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=null, leaderPayloadVersion=-1] 01:48:36,980 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:36,980 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:36,980 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Follower to Candidate 01:48:36,998 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)]] after [10 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 01:48:37,28 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 01:48:37,28 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-1679895222] 01:48:37,28 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:48:37,28 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Candidate, leaderId=null, activeMember=true] 01:48:37,158 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:48:37,158 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=6, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:37,158 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 6 01:48:37,158 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:37,248 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received ShardNotInitializedTimeout message for shard cars 01:48:37,248 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Returning NoShardLeaderException for shard cars 01:48:37,248 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: Find primary for shard cars failed org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:48:37,249 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Creating NoOpTransaction because of error org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:48:37,249 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] NoOpTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:48:37,249 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] NoOpTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 directCommit called, failure org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:48:37,249 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:48:37,250 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithCreateTxFailureDueToNoLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:48:37,250 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:48:37,250 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:48:37,250 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithCreateTxFailureDueToNoLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:48:37,251 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:48:37,251 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:48:37,251 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:48:37,251 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Aborting 0 pending queued transactions 01:48:37,252 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:48:37,252 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:37,252 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:37,680 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-9134258060952718266] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:38,270 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [925559536305288354] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:38,271 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:38,271 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:38,273 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:38.275] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:38,279 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:38,279 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:38,281 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:38.282] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:38,301 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:48:38,312 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [5535618924654989170] 01:48:38,312 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:48:38,314 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:48:38,314 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:48:38,314 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:38,314 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:38,326 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:48:38,326 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:48:38,326 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:48:38,331 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:38,340 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [6759168770147291494] 01:48:38,341 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:48:38,342 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:38,342 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:48:38,347 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:38,347 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:38,357 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 closeTransaction called 01:48:38,357 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_12#-99995868)] 01:48:38,357 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 closeTransaction called 01:48:38,357 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_10#-895769482)] 01:48:38,357 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 closeTransaction called 01:48:38,357 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_11#-1639406255)] 01:48:38,357 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 closeTransaction called 01:48:38,357 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_13#1701975856)] 01:48:38,392 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:38,411 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-3573827307985915800] 01:48:38,411 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:48:38,411 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:38,411 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:48:38,422 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:38,423 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:38,425 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithMultipleShards 01:48:38,444 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:48:38,453 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:38,465 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 01:48:38,467 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#733068601] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:38,472 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:38,472 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:38,480 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: no initial generation override, starting from 0 01:48:38,480 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:48:38,480 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@11c6af1d to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@302c12db 01:48:38,480 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@752e8aba 01:48:38,481 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, 0, 1710164918481, None)) 01:48:38,481 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithMultipleShards 01:48:38,481 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164918480,0,0)) 01:48:38,481 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:38,481 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:38,481 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@302c12db to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@14fb468a 01:48:38,481 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} started 01:48:38,482 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:38,485 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:38,487 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithMultipleShards 01:48:38,489 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:38,490 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:38,490 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:38,490 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1be277a4 01:48:38,490 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:38,490 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:38,490 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:38,490 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:38,490 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#271150554] 01:48:38,490 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:38,490 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Shard created : member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, persistent : true 01:48:38,491 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#271150554] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:38,491 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:38,491 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:38,491 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards on shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#1141952290] 01:48:38,492 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Starting recovery for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with journal batch size 1 01:48:38,492 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-people-testChainedTransactionFailureWithMultipleShards, persistent : true 01:48:38,493 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-people-testChainedTransactionFailureWithMultipleShards with journal batch size 1 01:48:38,493 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#413065014 created and ready for shard:member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:38,493 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-1697107687] 01:48:38,493 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#1141952290] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:38,493 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-2028563012] 01:48:38,497 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:38,497 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:38,497 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:48:38,497 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:48:38,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [cars] 01:48:38,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is initialized 01:48:38,497 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 01:48:38,508 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#413065014] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:38,508 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-1697107687] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:38,508 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-2028563012] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:38,508 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 01:48:38,508 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from null to Follower 01:48:38,512 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#2087716358 created and ready for shard:member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:38,512 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#2087716358] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:38,513 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:38,513 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:38,513 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [people] 01:48:38,513 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is initialized 01:48:38,514 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:48:38,514 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:48:38,515 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 01:48:38,515 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 01:48:38,515 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from null to Follower 01:48:38,521 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: no initial generation override, starting from 0 01:48:38,521 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:48:38,521 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7272df09 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@b0c155a 01:48:38,523 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@ee29b8d 01:48:38,523 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, 0, 1710164918521, None)) 01:48:38,523 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-619519039] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:38,523 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164918520,0,0)) 01:48:38,524 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 01:48:38,524 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@b0c155a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@124b1845 01:48:38,524 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} started 01:48:38,531 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithMultipleShards 01:48:38,531 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:38,531 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:38,531 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:38,531 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:38,531 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:38,531 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@72159274 01:48:38,531 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:38,531 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:38,532 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:38,532 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:38,532 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#-1419294327] 01:48:38,532 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:38,532 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:38,532 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:38,532 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards on shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards#-836702143] 01:48:38,532 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, persistent : true 01:48:38,532 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#-1419294327] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:38,533 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-1335472381 created and ready for shard:member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:38,533 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with journal batch size 1 01:48:38,533 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-people-testChainedTransactionFailureWithMultipleShards, persistent : true 01:48:38,533 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-people-testChainedTransactionFailureWithMultipleShards with journal batch size 1 01:48:38,534 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-1335472381] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:38,534 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards#-836702143] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:38,534 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#-49752675] 01:48:38,534 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#1182615416] 01:48:38,534 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#-49752675] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:38,535 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#1182615416] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:38,539 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#1745481802 created and ready for shard:member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:38,539 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#1745481802] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:38,541 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:38,541 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:38,542 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:48:38,542 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:38,542 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [people] 01:48:38,542 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards is initialized 01:48:38,542 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:38,542 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:38,549 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:48:38,549 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:48:38,549 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [cars] 01:48:38,549 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards is initialized 01:48:38,549 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:38,549 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 01:48:38,549 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 01:48:38,549 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-2-shard-people-testChainedTransactionFailureWithMultipleShards from null to Follower 01:48:38,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:48:38,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:48:38,552 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:38,556 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 01:48:38,556 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 01:48:38,556 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards from null to Follower 01:48:38,575 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#733068601] 01:48:38,575 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: findLocalShard : cars 01:48:38,575 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#271150554] 01:48:38,745 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:48:38,745 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:48:38,745 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testChainedTransactionFailureWithMultipleShards] 01:48:38,745 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:38,745 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 1 01:48:38,745 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:38,745 PM [cluster-test-shard-dispatcher-14] [DEBUG] MessageAssembler - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Closing 01:48:38,745 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:48:38,745 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:48:38,746 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from Follower to Candidate 01:48:38,746 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from Follower to Candidate 01:48:38,823 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:48:38,823 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:48:38,824 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testChainedTransactionFailureWithMultipleShards] 01:48:38,824 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:38,824 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 1 01:48:38,824 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:38,824 PM [cluster-test-shard-dispatcher-14] [DEBUG] MessageAssembler - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Closing 01:48:38,824 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:48:38,824 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:48:38,824 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from Follower to Candidate 01:48:38,824 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from Follower to Candidate 01:48:38,983 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:48:38,983 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:38,983 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 2 01:48:39,77 PM [cluster-test-shard-dispatcher-32] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:48:39,113 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:48:39,113 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:39,113 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 2 01:48:39,167 PM [cluster-test-shard-dispatcher-32] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:48:39,223 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:48:39,223 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:39,223 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 3 01:48:39,325 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:48:39,325 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:48:39,325 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:39,325 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:39,325 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:39,325 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:39,325 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:39,326 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:39,326 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:39,326 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:39,326 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:39,326 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:39,326 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:39,326 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Peer address for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:39,333 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:39,333 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:39,333 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:39,333 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:39,333 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:39,333 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:39,333 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:39,333 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:39,333 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:39,333 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:39,374 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:48:39,374 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:39,374 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 3 01:48:39,374 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:39,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:39,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Cluster state: [] 01:48:39,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:48:39,381 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:48:39,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=null 01:48:39,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 01:48:39,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:39,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 01:48:39,382 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 01:48:39,382 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 01:48:39,382 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testChainedTransactionFailureWithMultipleShards] 01:48:39,382 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:39,382 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 01:48:39,382 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:48:39,382 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: became leader with frontend state for [] 01:48:39,383 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from Candidate to Leader 01:48:39,383 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 01:48:39,383 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:39,383 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:39,383 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from Candidate to Leader 01:48:39,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Closing 01:48:39,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testChainedTransactionFailureWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:39,385 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 01:48:39,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:39,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:39,385 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-people-testChainedTransactionFailureWithMultipleShards status sync done false 01:48:39,386 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:48:39,386 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:39,431 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:48:39,494 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:48:39,494 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:39,494 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 4 01:48:39,494 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:39,496 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-people-testChainedTransactionFailureWithMultipleShards status sync done true 01:48:39,496 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:39,496 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Cluster state: [] 01:48:39,496 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:48:39,496 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:48:39,496 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=null 01:48:39,496 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:48:39,496 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:39,496 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:48:39,498 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:48:39,498 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:48:39,498 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testChainedTransactionFailureWithMultipleShards] 01:48:39,498 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:39,498 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:48:39,498 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:48:39,498 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: became leader with frontend state for [] 01:48:39,498 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from Candidate to Leader 01:48:39,498 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 01:48:39,498 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:39,498 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:39,498 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from Candidate to Leader 01:48:39,498 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All Shards are ready - data store testChainedTransactionFailureWithMultipleShards is ready 01:48:39,499 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Closing 01:48:39,499 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testChainedTransactionFailureWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:39,499 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 01:48:39,499 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:39,499 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:39,499 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All Shards are ready - data store testChainedTransactionFailureWithMultipleShards is ready 01:48:39,499 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards status sync done false 01:48:39,500 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:48:39,500 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:39,585 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#733068601] 01:48:39,585 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: findLocalShard : people 01:48:39,585 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#1141952290] 01:48:39,586 PM [main] [DEBUG] AbstractDOMBroker - Transaction chain 0 created with listener Mock for DOMTransactionChainListener, hashCode: 1551632157, backing store chains {CONFIGURATION=org.opendaylight.controller.cluster.datastore.TransactionChainProxy@3d76749f} 01:48:39,587 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testChainedTransactionFailureWithMultipleShards commit rateLimit adjusted to 40.88563184859886 01:48:39,587 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:39,587 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-619519039] 01:48:39,587 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:48:39,587 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:48:39,587 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 01:48:39,587 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-619519039] 01:48:39,587 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:48:39,587 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:48:39,587 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 01:48:39,588 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Readying 2 components for commit 01:48:39,588 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 01:48:39,588 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:48:39,588 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Readying transaction for shard people 01:48:39,588 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:48:39,588 PM [main] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 is submitted for execution. 01:48:39,588 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 01:48:39,588 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 canCommit 01:48:39,588 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 01:48:39,588 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards)] for shard people 01:48:39,588 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards)] found - creating WRITE_ONLY transaction context 01:48:39,588 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards)] for shard cars 01:48:39,588 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards)] found - creating WRITE_ONLY transaction context 01:48:39,588 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 01:48:39,588 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 = [ 1 ] 01:48:39,588 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction called 01:48:39,588 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:48:39,588 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 sending 1 batched modifications, ready: true 01:48:39,588 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards)] 01:48:39,588 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 = [ 1 ] 01:48:39,588 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction called 01:48:39,588 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 sending 1 batched modifications, ready: true 01:48:39,589 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards)] 01:48:39,591 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, disabling transaction tracking 01:48:39,591 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 01:48:39,591 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 01:48:39,591 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:48:39,591 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:39,591 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:39,591 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}]]] 01:48:39,592 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 01:48:39,592 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 01:48:39,592 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:48:39,592 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} 01:48:39,592 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@5f0135f2 01:48:39,592 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:39,592 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Readying Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 of 1 operations, client version 13 01:48:39,592 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: readying transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:39,592 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, disabling transaction tracking 01:48:39,592 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 ready 01:48:39,592 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE}}}]} 01:48:39,592 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 01:48:39,592 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 01:48:39,592 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:48:39,593 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:39,593 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:39,593 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}]]] 01:48:39,593 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 01:48:39,593 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 01:48:39,593 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:48:39,593 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} 01:48:39,593 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@19adf7b6 01:48:39,593 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 01:48:39,593 PM [cluster-test-shard-dispatcher-14] [ERROR] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Error handling BatchedModifications for Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:48:39,594 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 01:48:39,594 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 01:48:39,599 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 01:48:39,599 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 01:48:39,603 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}]]] 01:48:39,603 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 01:48:39,603 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:48:39,603 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 01:48:39,603 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 01:48:39,603 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction 01:48:39,603 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 1 01:48:39,603 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:39,603 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#1141952290)] resolved 01:48:39,603 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:48:39,603 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}]]] 01:48:39,603 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 01:48:39,603 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:48:39,603 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 01:48:39,603 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 01:48:39,603 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 1 01:48:39,603 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:39,604 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 01:48:39,604 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 01:48:39,604 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards status sync done true 01:48:39,605 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: a cohort Future failed org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:48:39,605 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 modifications failed org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:48:39,605 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during phase CAN_COMMIT, starting Abort org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:48:39,606 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:48:39,606 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:48:39,606 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 0 01:48:39,606 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 01:48:39,606 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 01:48:39,606 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:48:39,606 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:48:39,606 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, nanosAgo=15375011} is already disabled 01:48:39,607 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:39,607 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:39,607 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}]]] 01:48:39,607 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:48:39,607 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:48:39,607 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:48:39,607 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 0 01:48:39,607 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 01:48:39,607 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 01:48:39,607 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:48:39,607 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:48:39,607 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, nanosAgo=14779711} is already disabled 01:48:39,607 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:39,607 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DOMBrokerTransactionChain - Transaction chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@7ec3fa1e failed. TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema.]]} at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1119) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) at org.opendaylight.controller.cluster.datastore.ThreePhaseCommitCohortProxy$4.onFailure(ThreePhaseCommitCohortProxy.java:140) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1119) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) at org.opendaylight.controller.cluster.datastore.ThreePhaseCommitCohortProxy$3.onComplete(ThreePhaseCommitCohortProxy.java:102) at org.opendaylight.controller.cluster.datastore.ThreePhaseCommitCohortProxy$3.onComplete(ThreePhaseCommitCohortProxy.java:95) at akka.dispatch.OnComplete.internal(Future.scala:299) at akka.dispatch.OnComplete.internal(Future.scala:297) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:48:39,608 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 abort 01:48:39,608 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: a abort cohort path Future failed org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:48:39,608 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 aborted successfully 01:48:39,607 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:39,609 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}]]] 01:48:39,609 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}]]] 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 2 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 0 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 0 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}]]] 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 2 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 0 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 0 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:48:39,610 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:48:39,611 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 01:48:39,611 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:48:39,611 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 01:48:39,611 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:48:39,613 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:48:39,613 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:48:39,613 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 1 01:48:39,613 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 01:48:39,613 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:48:39,613 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 01:48:39,613 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:48:39,613 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} has no callback 01:48:39,613 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 1 01:48:39,613 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 01:48:39,613 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 01:48:39,613 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:48:39,613 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} has no callback 01:48:39,613 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 3 01:48:39,613 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:39,613 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:39,613 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:48:39,613 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:48:39,614 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 1 01:48:39,614 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:48:39,614 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 1 01:48:39,614 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 01:48:39,614 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 01:48:39,614 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:48:39,614 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:48:39,615 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 1 01:48:39,615 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:48:39,615 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 1 01:48:39,615 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 01:48:39,615 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 01:48:39,615 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 3 01:48:39,615 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:48:39,618 PM [main] [DEBUG] DOMBrokerTransactionChain - Chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@7ec3fa1e is no longer running 01:48:39,618 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:48:39,618 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:39,618 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:39,618 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards - waiting on 2 shards 01:48:39,618 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onShutDown 01:48:39,618 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Initiating leader transfer 01:48:39,619 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onShutDown 01:48:39,619 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Initiating leader transfer 01:48:39,619 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@392dc3e6 01:48:39,619 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@392dc3e6 01:48:39,619 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Attempting to transfer leadership 01:48:39,619 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:39,619 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:48:39,619 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:39,619 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:39,619 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:48:39,619 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:39,619 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:39,619 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4d76063f 01:48:39,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards] 01:48:39,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards] 01:48:39,621 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:48:39,621 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:39,621 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:39,621 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:48:39,621 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:39,621 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:39,621 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:48:39,621 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:48:39,621 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:39,622 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Leader transfer complete 01:48:39,833 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Attempting to transfer leadership 01:48:39,833 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:39,835 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testChainedTransactionFailureWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:48:39,835 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:48:39,835 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:39,836 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Leader transfer complete 01:48:40,34 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All shards shutdown - sending PoisonPill to self 01:48:40,34 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#271150554]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:48:40,34 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:40,34 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader)_slicer-id-467: Closing 01:48:40,34 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Stopping Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:40,34 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader)_slicer-id-468: Closing 01:48:40,34 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Aborting 1 pending queued transactions 01:48:40,34 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 01:48:40,35 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 01:48:40,35 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 01:48:40,35 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:48:40,35 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: shutting down 01:48:40,35 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 01:48:40,35 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 01:48:40,36 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#733068601] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#733068601] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:40,36 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#733068601] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#733068601] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:40,36 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#1972962730] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1972962730] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:40,36 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:40,37 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:40,47 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:40,47 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:40,47 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:40,47 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:40,49 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:48:40,64 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 01:48:40,64 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:48:40,64 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Closing 01:48:40,64 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:48:40,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Closing 01:48:40,64 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 01:48:40,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 01:48:40,65 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 01:48:40,65 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:40,65 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:41,51 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [5535618924654989170] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:41,80 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [6759168770147291494] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:41,80 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:41,80 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:41,82 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:41.084] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:41,89 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:41,89 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:41,90 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:41.091] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:41,114 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:41,120 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-6208009007646673434] 01:48:41,120 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:48:41,122 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:48:41,122 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:48:41,133 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:41,133 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:41,133 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:48:41,133 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:48:41,136 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:41,136 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:48:41,144 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [672590294564487563] 01:48:41,144 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:48:41,159 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:41,159 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:48:41,159 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:41,159 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:41,164 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:41,176 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-3346290611933047079] 01:48:41,176 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:48:41,178 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:41,178 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:48:41,190 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:41,190 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:41,194 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:41,195 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:41,196 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 01:48:41,201 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 01:48:41,206 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#434798008] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:41,222 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 01:48:41,222 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:48:41,222 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@273af59f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@456b676a 01:48:41,223 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5666f4d5 01:48:41,223 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1710164921223, None)) 01:48:41,223 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 01:48:41,223 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:41,223 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-1-shard-people-testLeadershipTransferOnShutdown 01:48:41,223 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164921222,0,0)) 01:48:41,223 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:41,223 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:41,223 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:48:41,223 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@11f7497 01:48:41,223 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@456b676a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@213d76ac 01:48:41,223 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 01:48:41,224 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:41,224 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:41,224 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:41,226 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:41,228 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 01:48:41,230 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:41,230 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:41,230 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-1-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#1919903376] 01:48:41,230 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testLeadershipTransferOnShutdown 01:48:41,231 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-1-shard-cars-testLeadershipTransferOnShutdown, persistent : true 01:48:41,231 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#1919903376] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:41,232 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#-1341713432 created and ready for shard:member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:41,232 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-1-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 01:48:41,232 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-1308676587] 01:48:41,232 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#-1341713432] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:41,232 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-1-shard-people-testLeadershipTransferOnShutdown, persistent : true 01:48:41,232 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-notifier#-1151840334 created and ready for shard:member-1-shard-people-testLeadershipTransferOnShutdown 01:48:41,232 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#1388167879] 01:48:41,237 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 01:48:41,238 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:48:41,238 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:48:41,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:48:41,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-1-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#926462505] 01:48:41,239 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#102027363] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:41,239 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-1308676587] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:41,240 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#926462505] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:41,240 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-notifier#-1151840334] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:41,240 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#1388167879] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:41,241 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:41,241 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:41,241 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 01:48:41,241 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-1-shard-cars-testLeadershipTransferOnShutdown is initialized 01:48:41,241 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:48:41,241 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:48:41,241 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 01:48:41,241 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:48:41,241 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from null to Follower 01:48:41,242 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-1-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 01:48:41,243 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:41,243 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:41,243 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 01:48:41,243 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-testLeadershipTransferOnShutdown is initialized 01:48:41,243 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:48:41,243 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:48:41,244 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 01:48:41,244 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:48:41,244 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from null to Follower 01:48:41,246 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:41,251 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 01:48:41,252 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:48:41,252 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7a4c929f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@72f56cae 01:48:41,252 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@37f7ec94 01:48:41,252 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 01:48:41,252 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:41,252 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-2-shard-people-testLeadershipTransferOnShutdown 01:48:41,252 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:41,252 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:41,252 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:48:41,255 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1710164921255, None)) 01:48:41,255 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164921254,0,0)) 01:48:41,255 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@72f56cae to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@219a1a52 01:48:41,256 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@57563507 01:48:41,258 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:41,262 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 01:48:41,262 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:48:41,262 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:41,262 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:41,263 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:41,263 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:41,263 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-2-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1838634964] 01:48:41,263 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testLeadershipTransferOnShutdown 01:48:41,263 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Shard created : member-2-shard-cars-testLeadershipTransferOnShutdown, persistent : true 01:48:41,263 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1838634964] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:41,264 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Starting recovery for member-2-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 01:48:41,264 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Shard created : member-2-shard-people-testLeadershipTransferOnShutdown, persistent : true 01:48:41,264 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-notifier#1240815455 created and ready for shard:member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:41,264 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#1056464545] 01:48:41,264 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#1479741165] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:41,264 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-notifier#1240815455] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:41,264 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#1056464545] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:41,264 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-notifier#-244822194 created and ready for shard:member-2-shard-people-testLeadershipTransferOnShutdown 01:48:41,264 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#489837869] 01:48:41,264 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-notifier#-244822194] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:41,264 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Starting recovery for member-2-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 01:48:41,264 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#489837869] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:41,265 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:48:41,265 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:48:41,265 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-2-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#1479741165] 01:48:41,267 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:41,267 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:41,267 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:41,267 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:41,267 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:48:41,267 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:48:41,267 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:48:41,267 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:48:41,267 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 01:48:41,267 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-1-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:48:41,267 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 01:48:41,267 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:41,267 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 01:48:41,267 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized 01:48:41,267 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 01:48:41,267 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is initialized 01:48:41,267 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:48:41,267 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:48:41,267 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from null to Follower 01:48:41,267 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from null to Follower 01:48:41,313 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#434798008] 01:48:41,313 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 01:48:41,321 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#1919903376] 01:48:41,545 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:48:41,545 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 01:48:41,545 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-3-shard-cars-testLeadershipTransferOnShutdown] 01:48:41,545 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:41,545 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 01:48:41,545 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:41,545 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 01:48:41,545 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:48:41,545 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:48:41,545 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:48:41,545 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 01:48:41,555 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:48:41,555 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 01:48:41,555 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 01:48:41,555 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 01:48:41,555 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 01:48:41,555 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:41,555 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 01:48:41,555 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:48:41,555 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:48:41,556 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:48:41,556 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 01:48:41,775 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:48:41,775 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=2, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:41,775 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 2 01:48:41,804 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:48:41,805 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=2, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 01:48:41,805 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 2 01:48:42,4 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:48:42,4 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:42,5 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 3 01:48:42,125 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:48:42,125 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 01:48:42,125 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 3 01:48:42,146 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:48:42,146 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:48:42,147 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:42,147 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,147 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:42,147 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:48:42,147 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:48:42,147 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:42,147 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-3-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,147 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:42,147 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:42,147 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:48:42,147 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:48:42,147 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:42,147 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-3-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:48:42,147 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-2-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:48:42,154 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:42,154 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,154 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:42,154 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:48:42,154 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:48:42,154 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:42,154 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:42,154 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:42,154 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:48:42,154 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:48:42,154 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-3-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,154 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-3-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:48:42,200 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:48:42,254 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:48:42,254 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:42,255 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 4 01:48:42,255 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:42,255 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,256 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:42,256 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 01:48:42,256 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:48:42,256 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:48:42,256 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 01:48:42,257 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:48:42,257 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:42,257 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:48:42,257 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#1919903376] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:42,258 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:48:42,258 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:48:42,258 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-3-shard-cars-testLeadershipTransferOnShutdown] 01:48:42,259 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:42,259 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:42,259 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:48:42,259 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 01:48:42,259 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [] 01:48:42,259 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 01:48:42,260 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 01:48:42,260 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:48:42,260 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:42,260 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:42,260 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:42,260 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 01:48:42,260 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#1919903376] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:42,260 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:48:42,260 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:42,260 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:42,260 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done false 01:48:42,261 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 01:48:42,261 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:42,313 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 01:48:42,313 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 01:48:42,313 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 01:48:42,313 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 01:48:42,313 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 01:48:42,314 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=1, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-people-testLeadershipTransferOnShutdown 01:48:42,314 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=1, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-testLeadershipTransferOnShutdown 01:48:42,314 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:42,314 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 01:48:42,314 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:48:42,314 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:48:42,315 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:48:42,315 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 01:48:42,316 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVote [term=1, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1], my term is 3 01:48:42,316 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): In requestVote: RequestVote [term=1, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: member-1-shard-people-testLeadershipTransferOnShutdown, lastIndex: -1, lastTerm: -1 01:48:42,316 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): requestVote returning: RequestVoteReply [term=3, voteGranted=false] 01:48:42,317 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=false], my term is 1 01:48:42,317 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Term 3 in "RequestVoteReply [term=3, voteGranted=false]" message is greater than Candidate's term 1 - switching to Follower 01:48:42,318 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=null 01:48:42,318 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Follower, election term: 3 01:48:42,318 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:48:42,318 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:48:42,318 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Follower 01:48:42,318 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Candidate to Follower 01:48:42,319 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#1479741165] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:42,327 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#434798008] 01:48:42,327 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : people 01:48:42,327 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#926462505] 01:48:42,365 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:48:42,365 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 01:48:42,365 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 4 01:48:42,365 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testLeadershipTransferOnShutdown 01:48:42,365 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-testLeadershipTransferOnShutdown 01:48:42,368 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#926462505] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:42,368 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:42,368 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 01:48:42,368 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:48:42,368 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 3 - updating term 01:48:42,368 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 01:48:42,368 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:48:42,368 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 01:48:42,368 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:48:42,370 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:48:42,370 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:48:42,370 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 01:48:42,370 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:42,370 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:42,370 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:48:42,370 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 01:48:42,370 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: became leader with frontend state for [] 01:48:42,370 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 01:48:42,370 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:48:42,370 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:42,370 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:42,371 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Candidate to Leader 01:48:42,371 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:48:42,373 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#926462505] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:42,374 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown: Closing 01:48:42,374 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:48:42,374 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:48:42,374 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:42,374 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:42,374 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:48:42,374 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-people-testLeadershipTransferOnShutdown status sync done false 01:48:42,375 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 01:48:42,375 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:42,376 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done true 01:48:42,376 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#1919903376] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:42,378 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:42,379 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:42,381 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 01:48:42,386 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1091781401] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:42,387 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 01:48:42,387 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 01:48:42,387 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:48:42,388 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@c5d332a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@533d3fdd 01:48:42,388 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2403267e 01:48:42,388 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 01:48:42,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,388 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1710164922388, None)) 01:48:42,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-3-shard-people-testLeadershipTransferOnShutdown 01:48:42,388 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164922387,0,0)) 01:48:42,388 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@533d3fdd to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@230f40c1 01:48:42,388 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:42,388 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,388 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:48:42,388 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 01:48:42,388 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:42,388 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:42,388 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:48:42,388 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:42,388 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:42,388 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:48:42,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7a7cf7d8 01:48:42,389 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testLeadershipTransferOnShutdown commit rateLimit adjusted to 40.88563184859886 01:48:42,389 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,389 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#102027363] 01:48:42,389 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:48:42,389 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 01:48:42,389 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 01:48:42,389 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 01:48:42,389 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#102027363] 01:48:42,389 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 01:48:42,389 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:48:42,389 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Readying 2 components for commit 01:48:42,389 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Readying transaction for shard cars 01:48:42,389 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 01:48:42,389 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown, primaryVersion=13] 01:48:42,389 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 01:48:42,389 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Readying transaction for shard people 01:48:42,389 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 01:48:42,390 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] for shard cars 01:48:42,390 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] found - creating WRITE_ONLY transaction context 01:48:42,390 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:48:42,390 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:48:42,390 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 = [ 2 ] 01:48:42,390 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 readyTransaction called 01:48:42,390 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 sending 2 batched modifications, ready: true 01:48:42,390 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] 01:48:42,390 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown, primaryVersion=13] 01:48:42,390 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown)] for shard people 01:48:42,390 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown)] found - creating WRITE_ONLY transaction context 01:48:42,390 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 01:48:42,390 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 = [ 1 ] 01:48:42,390 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 readyTransaction called 01:48:42,390 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 sending 1 batched modifications, ready: true 01:48:42,390 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown)] 01:48:42,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:42,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,390 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#434798008] 01:48:42,391 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:42,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#1491292024] 01:48:42,391 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:42,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-2-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#1491292024] 01:48:42,391 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-3-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#1491292024] 01:48:42,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-people-testLeadershipTransferOnShutdown 01:48:42,391 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-3-shard-cars-testLeadershipTransferOnShutdown, persistent : true 01:48:42,391 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 01:48:42,391 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#1491292024] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:42,391 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:48:42,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:48:42,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#-1227949064] 01:48:42,391 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#1919903376] 01:48:42,391 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:48:42,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:48:42,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-2-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#-1227949064] 01:48:42,391 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:48:42,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:48:42,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-3-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#-1227949064] 01:48:42,391 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#1919903376] 01:48:42,392 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-notifier#1021522858 created and ready for shard:member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,392 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-3-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 01:48:42,392 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-3-shard-people-testLeadershipTransferOnShutdown, persistent : true 01:48:42,392 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#-498459875] 01:48:42,392 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#-1227949064] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:42,392 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-3-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 01:48:42,393 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#-606467375] 01:48:42,393 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-notifier#1021522858] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:42,393 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-notifier#1120492585 created and ready for shard:member-3-shard-people-testLeadershipTransferOnShutdown 01:48:42,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, disabling transaction tracking 01:48:42,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:48:42,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:48:42,394 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, disabling transaction tracking 01:48:42,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:48:42,394 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:48:42,394 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:48:42,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:42,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:42,394 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:48:42,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 01:48:42,394 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:42,394 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 01:48:42,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:42,394 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 01:48:42,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying 2 batched modifications for Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,394 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:42,394 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying 1 batched modifications for Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,394 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Readying Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 of 1 operations, client version 13 01:48:42,394 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,394 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 ready 01:48:42,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Readying Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 of 2 operations, client version 13 01:48:42,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,395 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 ready 01:48:42,395 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:42,395 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:42,395 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:42,395 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:42,395 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:48:42,395 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:48:42,395 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:48:42,395 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:48:42,395 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:48:42,395 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:42,395 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:48:42,395 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:42,395 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 01:48:42,395 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 01:48:42,395 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-cars-testLeadershipTransferOnShutdown is initialized 01:48:42,395 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 01:48:42,395 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:48:42,395 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:48:42,395 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 01:48:42,395 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:48:42,395 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:48:42,395 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:48:42,395 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:48:42,396 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 1 01:48:42,396 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 readyTransaction 01:48:42,396 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:42,396 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 01:48:42,396 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:48:42,396 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:48:42,396 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 01:48:42,396 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:48:42,396 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 01:48:42,396 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:48:42,396 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 1 01:48:42,396 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:42,396 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 01:48:42,396 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:42,396 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 01:48:42,396 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 01:48:42,396 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-3-shard-people-testLeadershipTransferOnShutdown: Closing 01:48:42,396 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:48:42,396 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:48:42,396 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:48:42,396 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:48:42,396 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-people-testLeadershipTransferOnShutdown status sync done true 01:48:42,397 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 01:48:42,397 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-people-testLeadershipTransferOnShutdown is initialized 01:48:42,397 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 01:48:42,397 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from null to Follower 01:48:42,397 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:48:42,397 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:42,397 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:42,397 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:42,397 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 01:48:42,397 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:48:42,397 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Setting last applied to 0 01:48:42,397 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:48:42,397 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-people-testLeadershipTransferOnShutdown status sync done false 01:48:42,397 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:48:42,397 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:48:42,397 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:48:42,397 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-1-shard-people-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 01:48:42,397 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=3348835} is already disabled 01:48:42,397 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:48:42,397 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 0 01:48:42,397 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:42,397 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:48:42,397 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:48:42,397 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:48:42,397 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-cars-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 01:48:42,397 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=3592208} is already disabled 01:48:42,397 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:48:42,397 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:48:42,398 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 01:48:42,398 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:42,398 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,398 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 01:48:42,398 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 01:48:42,398 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:42,398 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 01:48:42,398 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 01:48:42,399 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 readyTransaction 01:48:42,399 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 01:48:42,399 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:48:42,399 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:48:42,399 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:48:42,399 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 01:48:42,399 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:48:42,399 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:48:42,399 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:48:42,399 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-people-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 01:48:42,399 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:48:42,399 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:48:42,399 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:42,399 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:42,399 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-people-testLeadershipTransferOnShutdown from null to Follower 01:48:42,399 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:48:42,400 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done false 01:48:42,400 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 01:48:42,400 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:48:42,400 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:48:42,400 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:48:42,400 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:48:42,400 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 1 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendMetadata - member-3-shard-cars-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 1 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:48:42,401 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendMetadata - member-3-shard-people-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 01:48:42,402 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:48:42,402 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:48:42,402 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:48:42,402 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:48:42,402 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-people-testLeadershipTransferOnShutdown status sync done true 01:48:42,402 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done true 01:48:42,403 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 01:48:42,403 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 01:48:42,404 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:42,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:42,442 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#1919903376] 01:48:42,443 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:48:42,443 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:42,443 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:48:42,443 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] found - creating WRITE_ONLY transaction context 01:48:42,443 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 01:48:42,443 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 Readying 1 components for commit 01:48:42,443 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 Readying transaction for shard cars 01:48:42,443 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 directCommit called 01:48:42,443 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 sending 1 batched modifications, ready: true 01:48:42,443 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] 01:48:42,444 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#434798008] 01:48:42,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 01:48:42,444 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#1919903376] 01:48:42,444 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#1919903376] 01:48:42,453 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying 1 batched modifications for Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:48:42,453 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Readying Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 of 1 operations, client version 13 01:48:42,453 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:48:42,453 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 ready 01:48:42,453 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 is scheduled for canCommit step 01:48:42,494 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#1919903376] 01:48:42,495 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testLeadershipTransferOnShutdown 01:48:42,495 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#434798008] 01:48:42,495 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:42,496 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 01:48:42,496 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-people-testLeadershipTransferOnShutdown 01:48:42,496 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 01:48:42,496 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 01:48:42,496 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#1919903376] 01:48:42,496 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 canCommit 01:48:42,496 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onShutDown 01:48:42,496 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Initiating leader transfer 01:48:42,496 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3c2c5906 01:48:42,496 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#1919903376)] resolved 01:48:42,496 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#926462505)] resolved 01:48:42,496 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: successfully resolved all cohort actors 01:48:42,496 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 finishCanCommit 01:48:42,497 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#1919903376)] 01:48:42,497 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#1919903376)] 01:48:42,497 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:48:42,497 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:42,497 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:42,498 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown] 01:48:42,498 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown] 01:48:42,498 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:48:42,498 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:42,498 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:42,499 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Can committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,499 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:48:42,499 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,499 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 validated 01:48:42,499 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:42,499 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: canCommit for member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: success 01:48:42,499 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:42,499 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:42,500 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:48:42,500 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#926462505)] 01:48:42,500 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#926462505)] 01:48:42,501 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Can committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,501 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,501 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 validated 01:48:42,501 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:42,501 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: canCommit for member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: success 01:48:42,501 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:48:42,501 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit returning result: true 01:48:42,501 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 commit 01:48:42,501 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 finish commit 01:48:42,502 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#1919903376)] 01:48:42,502 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#1919903376)] 01:48:42,502 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#926462505)] 01:48:42,502 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#926462505)] 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 candidate ready 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 candidate ready 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit - messages: [] 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit - messages: [] 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 prepared 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 prepared 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Finishing commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Finishing commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 173 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 169 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 validated 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: canCommit for member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: success 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 candidate ready 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: canCommit - candidate rootPath: / 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: canCommit - messages: [] 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 prepared 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Finishing commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 scheduled for commit step 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: false 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 submitted to persistence 01:48:42,503 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 01:48:42,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 submitted to persistence 01:48:42,504 PM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 228 01:48:42,504 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 01:48:42,504 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 01:48:42,504 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:42,504 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:48:42,504 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:42,504 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 01:48:42,504 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:48:42,504 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,504 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 01:48:42,504 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 submitted to persistence 01:48:42,505 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:48:42,505 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 01:48:42,505 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=173} 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 2 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testLeadershipTransferOnShutdown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 2 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=169} 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=228} 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 3 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 2 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:48:42,506 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-cars-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 01:48:42,506 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 01:48:42,506 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:48:42,507 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:48:42,507 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=173} 01:48:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 01:48:42,507 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:48:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 2 01:48:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:48:42,507 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 2 01:48:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=169} 01:48:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 01:48:42,507 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-testLeadershipTransferOnShutdown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=228} 01:48:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 01:48:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 3 01:48:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:42,507 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 01:48:42,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 01:48:42,507 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:48:42,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 01:48:42,507 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Setting last applied to 1 01:48:42,507 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:48:42,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 1 01:48:42,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:48:42,507 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=173} 01:48:42,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=169} 01:48:42,507 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:48:42,507 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:48:42,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:48:42,507 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:48:42,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testLeadershipTransferOnShutdown_member-1-shard-cars-testLeadershipTransferOnShutdown%231919903376$f] 01:48:42,507 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testLeadershipTransferOnShutdown_member-1-shard-people-testLeadershipTransferOnShutdown%23926462505$g] 01:48:42,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-2003297965] with name member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 01:48:42,507 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1764349811] with name member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 01:48:42,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 2 01:48:42,507 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 01:48:42,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 01:48:42,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=228} 01:48:42,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:48:42,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: commit - successfulFromPrevious: [] 01:48:42,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 01:48:42,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testLeadershipTransferOnShutdown_member-1-shard-cars-testLeadershipTransferOnShutdown$c] 01:48:42,507 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:42,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3c2c5906 01:48:42,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 01:48:42,507 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 01:48:42,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:48:42,508 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:48:42,508 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,508 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1764349811] 01:48:42,508 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 01:48:42,508 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:48:42,508 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 01:48:42,508 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:48:42,508 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testLeadershipTransferOnShutdown, matchIndex: 2, lastIndex: 2, isVoting: true 01:48:42,508 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 01:48:42,508 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:48:42,508 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 91.88 μs 01:48:42,508 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:48:42,508 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,508 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 01:48:42,508 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 01:48:42,508 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1764349811] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:42,508 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-2003297965] 01:48:42,508 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:48:42,508 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 39.44 μs 01:48:42,508 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:48:42,508 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 30.30 μs 01:48:42,508 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-2003297965] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:42,509 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: commit succeeded 01:48:42,510 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 modifications completed with CommitTransactionReply [version=13] 01:48:42,510 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 01:48:42,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:48:42,510 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 01:48:42,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 1 01:48:42,510 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:48:42,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:48:42,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 01:48:42,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:48:42,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=173} 01:48:42,510 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 2 01:48:42,510 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 01:48:42,510 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 01:48:42,510 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:48:42,510 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=169} 01:48:42,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,510 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,510 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 01:48:42,510 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:42,511 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 01:48:42,511 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 1 for term 4 01:48:42,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1136446143] with name member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 01:48:42,511 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:48:42,511 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-2059385493] with name member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 01:48:42,511 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1136446143] 01:48:42,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 2 01:48:42,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 01:48:42,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=228} 01:48:42,511 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 01:48:42,511 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:48:42,511 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 01:48:42,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:48:42,511 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 81.61 μs 01:48:42,511 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-2059385493] 01:48:42,511 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:48:42,511 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1136446143] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:42,511 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-2059385493] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:42,511 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 39.72 μs 01:48:42,511 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 canCommit 01:48:42,511 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:48:42,511 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 35.32 μs 01:48:42,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 01:48:42,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 01:48:42,511 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 successfully completed direct commit 01:48:42,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 01:48:42,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:42,511 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 5 01:48:42,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,512 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] to peer member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:42,512 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:48:42,512 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 01:48:42,512 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:48:42,512 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:48:42,512 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 01:48:42,512 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:48:42,512 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 01:48:42,512 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 01:48:42,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 01:48:42,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 01:48:42,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 01:48:42,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 2 01:48:42,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] is present: existingEntryTerm: 4 01:48:42,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] is present: existingEntryTerm: 4 01:48:42,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 2, entries to be added from: 2 01:48:42,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 3 01:48:42,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 2 01:48:42,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:42,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 01:48:42,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 01:48:42,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:48:42,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=169} 01:48:42,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#739201683] with name member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 01:48:42,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 2 01:48:42,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 01:48:42,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=228} 01:48:42,514 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:48:42,514 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 01:48:42,514 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:48:42,514 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#739201683] 01:48:42,514 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:48:42,514 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 51.30 μs 01:48:42,514 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:48:42,515 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 30.87 μs 01:48:42,515 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:42,515 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 01:48:42,515 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 01:48:42,515 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 01:48:42,515 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4]" message is greater than leader's term 4 - switching to Follower 01:48:42,515 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 01:48:42,515 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 01:48:42,515 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 2, lastTerm: 4 01:48:42,515 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:42,515 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:48:42,515 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 5 01:48:42,515 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-475: Closing 01:48:42,515 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:48:42,515 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:48:42,515 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: removing frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 01:48:42,515 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown: Closing 01:48:42,515 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Leader to Follower 01:48:42,516 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 01:48:42,516 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Leader to Follower 01:48:42,517 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:48:42,517 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:48:42,517 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 01:48:42,517 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:48:42,517 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:48:42,517 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:48:42,517 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 01:48:42,517 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 01:48:42,517 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 01:48:42,517 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:48:42,517 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:42,517 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:42,517 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 01:48:42,517 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:48:42,518 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:42,518 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 01:48:42,518 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 01:48:42,518 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 1 01:48:42,518 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 4 - updating term 01:48:42,518 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 01:48:42,518 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:48:42,518 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 01:48:42,518 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 2, lastTerm: 4 01:48:42,518 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:48:42,518 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:42,518 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=173} 01:48:42,518 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:48:42,518 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-3-shard-people-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,518 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown: Closing 01:48:42,518 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:42,518 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1962094246] with name member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 01:48:42,518 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 01:48:42,518 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 01:48:42,519 PM [cluster-test-shard-dispatcher-28] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-2-shard-cars-testLeadershipTransferOnShutdown in 22.39 ms 01:48:42,519 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 01:48:42,519 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:48:42,519 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:42,519 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:42,519 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:48:42,519 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-1-shard-cars-testLeadershipTransferOnShutdown status sync done false 01:48:42,519 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1962094246] 01:48:42,519 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Stopping Shard member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:42,519 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 01:48:42,519 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:48:42,519 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:48:42,519 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 65.61 μs 01:48:42,519 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 01:48:42,519 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 01:48:42,519 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:42,519 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#102027363] 01:48:42,519 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 Queuing TransactionOperation 01:48:42,519 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done false 01:48:42,520 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:48:42,520 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:42,520 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:42,520 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:48:42,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:48:42,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1838634964, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6fdcd060] 01:48:42,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 01:48:42,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:48:42,520 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1838634964, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6fdcd060] 01:48:42,520 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1838634964)] for shard cars 01:48:42,520 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 01:48:42,520 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@45709472 01:48:42,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 01:48:42,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 1 for term 5 01:48:42,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:48:42,521 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,521 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1091781401] 01:48:42,521 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 01:48:42,521 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:48:42,521 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 01:48:42,521 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 1 for term 5 01:48:42,521 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1838634964] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:42,521 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done true 01:48:42,521 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 01:48:42,521 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractTransactionContextFactory - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown)] for shard cars 01:48:42,521 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown)] found - trying create transaction 01:48:42,521 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown)] 01:48:42,522 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: encountered ask-based client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, disabling transaction tracking 01:48:42,522 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:48:42,522 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:48:42,522 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:48:42,522 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 01:48:42,522 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 3 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,522 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 01:48:42,522 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 01:48:42,522 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:42,522 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-1-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 01:48:42,522 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Creating transaction : member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:48:42,522 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-3:datastore-testLeadershipTransferOnShutdown@0:0_14#-5413732] 01:48:42,523 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-3:datastore-testLeadershipTransferOnShutdown@0:0_14#-5413732] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:42,523 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 01:48:42,523 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:48:42,523 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:48:42,523 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:48:42,523 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:48:42,523 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 2 01:48:42,523 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=3, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:42,523 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-3:datastore-testLeadershipTransferOnShutdown@0:0_14#-5413732, transactionId=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=13] 01:48:42,523 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:48:42,523 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-3:datastore-testLeadershipTransferOnShutdown@0:0_14#-5413732)] 01:48:42,524 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 3, nextIndex: 4 01:48:42,524 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 1 01:48:42,525 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 ReadData operation succeeded 01:48:42,525 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:48:42,525 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 01:48:42,525 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 3 01:48:42,525 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 2 to 3 01:48:42,525 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:48:42,525 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 3 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:48:42,525 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:48:42,525 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-cars-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 01:48:42,525 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: leader state Disabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=3406086} is already disabled 01:48:42,525 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 2 for term 5 01:48:42,525 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=3 01:48:42,525 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,525 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-3-shard-people-testLeadershipTransferOnShutdown 01:48:42,525 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 01:48:42,525 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onShutDown 01:48:42,526 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - Stopping Shard member-3-shard-people-testLeadershipTransferOnShutdown 01:48:42,526 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 01:48:42,526 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:48:42,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:48:42,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onShutDown 01:48:42,527 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - Stopping Shard member-3-shard-cars-testLeadershipTransferOnShutdown 01:48:42,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 01:48:42,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:48:42,527 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 01:48:42,527 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 01:48:42,527 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:48:42,528 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:48:42,528 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-1-shard-cars-testLeadershipTransferOnShutdown 01:48:42,528 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 01:48:42,528 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-3-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:48:42,528 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-1-shard-people-testLeadershipTransferOnShutdown 01:48:42,528 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 01:48:42,528 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onShutDown 01:48:42,528 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Initiating leader transfer 01:48:42,528 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:48:42,528 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:42,528 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:42,528 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@210a3a8c 01:48:42,528 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@210a3a8c 01:48:42,528 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 01:48:42,528 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:42,528 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:42,529 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown] 01:48:42,529 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:48:42,529 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:42,529 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:42,530 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testLeadershipTransferOnShutdown, matchIndex: 1, lastIndex: 1, isVoting: true 01:48:42,530 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 01:48:42,530 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:42,530 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:42,530 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 01:48:42,531 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 01:48:42,531 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 01:48:42,531 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 01:48:42,531 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 5 01:48:42,531 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-people-testLeadershipTransferOnShutdown 01:48:42,531 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] to peer member-3-shard-people-testLeadershipTransferOnShutdown 01:48:42,531 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:48:42,531 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 01:48:42,531 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:48:42,531 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:48:42,531 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown: Closing 01:48:42,531 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:48:42,531 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 01:48:42,532 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 01:48:42,533 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:42,533 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 01:48:42,533 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 01:48:42,533 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4]" message is greater than leader's term 4 - switching to Follower 01:48:42,533 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 01:48:42,533 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 01:48:42,533 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 01:48:42,533 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 01:48:42,533 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:48:42,533 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 5 01:48:42,533 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-people-testLeadershipTransferOnShutdown (Leader)_slicer-id-476: Closing 01:48:42,533 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:48:42,533 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:48:42,533 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: removing frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 01:48:42,533 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown: Closing 01:48:42,533 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Leader to Follower 01:48:42,533 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 01:48:42,533 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Leader to Follower 01:48:42,534 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:48:42,534 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:48:42,534 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 01:48:42,534 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:42,534 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:42,534 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:48:42,534 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 01:48:42,534 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: became leader with frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 01:48:42,534 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 01:48:42,534 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:48:42,534 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:42,534 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:42,534 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Candidate to Leader 01:48:42,534 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:48:42,535 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown: Closing 01:48:42,535 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:48:42,535 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:48:42,535 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:42,535 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:42,535 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:48:42,535 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-2-shard-people-testLeadershipTransferOnShutdown in 7.004 ms 01:48:42,535 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 01:48:42,535 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-1-shard-people-testLeadershipTransferOnShutdown status sync done false 01:48:42,535 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Stopping Shard member-1-shard-people-testLeadershipTransferOnShutdown 01:48:42,535 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 01:48:42,535 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:48:42,535 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 01:48:42,535 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 01:48:42,536 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:48:42,536 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:48:42,536 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 01:48:42,536 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 01:48:42,536 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:48:42,536 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:42,536 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:48:42,537 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:42,537 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:42,543 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:42,543 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:42,543 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:42,543 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:42,557 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:48:42,557 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-2-shard-cars-testLeadershipTransferOnShutdown 01:48:42,557 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-480: Closing 01:48:42,558 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:48:42,558 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-2-shard-people-testLeadershipTransferOnShutdown 01:48:42,558 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-2-shard-people-testLeadershipTransferOnShutdown (Leader)_slicer-id-481: Closing 01:48:42,558 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:48:42,558 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 01:48:42,558 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:42,558 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:43,559 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-6208009007646673434] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:43,569 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [672590294564487563] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:43,576 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:43,576 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:43,577 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:43.579] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:43,584 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:43,584 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:43,585 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:43.587] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:43,598 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:43,606 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4437841914933822848] 01:48:43,606 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:48:43,620 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:48:43,620 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:48:43,620 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:43,620 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:43,623 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:48:43,623 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:48:43,623 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:48:43,633 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:48:43,639 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-5251907920139704957] 01:48:43,639 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:48:43,646 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:43,646 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:48:43,651 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:43,651 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:43,666 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:43,692 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [7252078257978949757] 01:48:43,692 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:48:43,692 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:43,692 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:48:43,700 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:43,701 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:43,703 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionForwardedToLeaderAfterRetry 01:48:43,708 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 01:48:43,710 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:43,710 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:43,710 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1305921173] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:43,722 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionForwardedToLeaderAfterRetry 01:48:43,722 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: no initial generation override, starting from 0 01:48:43,722 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:43,722 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:48:43,722 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6d532ed0 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@78caa059 01:48:43,722 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1235146a 01:48:43,722 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:43,722 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:43,722 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:43,722 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:43,724 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, 0, 1710164923723, None)) 01:48:43,724 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164923722,0,0)) 01:48:43,725 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@8bda30b 01:48:43,725 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@78caa059 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@750b7470 01:48:43,725 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} started 01:48:43,725 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:43,727 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:43,728 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionForwardedToLeaderAfterRetry 01:48:43,729 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:43,729 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:43,730 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:43,730 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:43,730 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:43,730 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:43,730 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, persistent : true 01:48:43,730 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:43,730 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:43,730 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1652989729] 01:48:43,731 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:43,731 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, persistent : true 01:48:43,731 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-1037482376 created and ready for shard:member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:43,731 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#-2087781517] 01:48:43,731 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 01:48:43,731 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1652989729] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:43,731 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-1037482376] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:43,731 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#-2087781517] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:43,731 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#243914430 created and ready for shard:member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:43,731 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#243914430] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:43,731 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#620087247] 01:48:43,731 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#620087247] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:43,732 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:43,732 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:43,732 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [cars] 01:48:43,732 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is initialized 01:48:43,732 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:48:43,732 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:48:43,732 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 01:48:43,732 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 01:48:43,732 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from null to Follower 01:48:43,733 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 01:48:43,733 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:43,733 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:43,733 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:48:43,733 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:48:43,733 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 01:48:43,733 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [people] 01:48:43,733 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is initialized 01:48:43,733 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 01:48:43,733 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from null to Follower 01:48:43,739 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1603100816] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:43,739 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 01:48:43,741 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: no initial generation override, starting from 0 01:48:43,741 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:48:43,741 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@386b37a4 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@12a0b177 01:48:43,741 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@169f37fb 01:48:43,741 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionForwardedToLeaderAfterRetry 01:48:43,741 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:43,741 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:43,741 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, 0, 1710164923741, None)) 01:48:43,742 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164923740,0,0)) 01:48:43,742 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@12a0b177 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@62840a79 01:48:43,742 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} started 01:48:43,742 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6f248009 01:48:43,742 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:43,742 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:43,743 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:43,743 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, persistent : true 01:48:43,744 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 01:48:43,744 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, persistent : true 01:48:43,744 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 01:48:43,745 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-551065751 created and ready for shard:member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:43,745 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#1829020989] 01:48:43,745 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-150494022 created and ready for shard:member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:43,745 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-194129467] 01:48:43,745 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1341379471] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:43,745 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#1269956772] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:43,745 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-551065751] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:43,745 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#1829020989] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:43,745 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-150494022] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:43,745 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-194129467] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:43,747 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:43,747 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:43,754 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [people] 01:48:43,754 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is initialized 01:48:43,755 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:43,755 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:43,755 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [cars] 01:48:43,755 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is initialized 01:48:43,755 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:48:43,755 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:48:43,755 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 01:48:43,755 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:48:43,755 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:48:43,755 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 01:48:43,755 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 01:48:43,755 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 01:48:43,755 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from null to Follower 01:48:43,755 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from null to Follower 01:48:43,761 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:48:43,761 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:48:43,762 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:43,765 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:43,765 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:43,765 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:43,765 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:43,765 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:43,765 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:43,765 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:43,766 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:43,792 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1305921173] 01:48:43,793 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:48:43,793 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:43,965 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:48:43,965 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Received ElectionTimeout - switching to Candidate 01:48:43,965 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 01:48:43,965 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=1, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:43,965 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 1 01:48:43,965 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:43,965 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:48:43,965 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:48:43,965 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:48:43,965 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 01:48:43,965 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 01:48:44,24 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:48:44,24 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Received ElectionTimeout - switching to Candidate 01:48:44,24 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 01:48:44,24 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:44,24 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 1 01:48:44,24 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:44,24 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:48:44,24 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:48:44,24 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:48:44,25 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 01:48:44,25 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 01:48:44,284 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:48:44,284 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=2, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:44,284 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 2 01:48:44,293 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:48:44,313 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:48:44,314 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:48:44,314 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:44,314 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 2 01:48:44,504 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:48:44,504 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:44,504 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 3 01:48:44,544 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:48:44,544 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:44,544 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 3 01:48:44,626 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:44,626 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:44,626 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:44,626 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:44,626 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:44,626 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:44,626 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:44,626 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:44,626 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:44,626 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:44,626 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:44,626 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:44,626 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:48:44,626 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:48:44,681 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:44,681 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:44,681 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:44,681 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:44,681 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:44,681 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:44,681 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:44,681 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:44,681 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:44,681 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:44,720 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:48:44,764 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:48:44,764 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:44,764 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 4 01:48:44,764 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:44,765 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:44,765 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 01:48:44,765 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:48:44,765 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:48:44,765 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=null 01:48:44,766 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:48:44,766 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:44,766 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:48:44,767 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:48:44,767 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:48:44,767 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 01:48:44,767 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:44,767 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:48:44,768 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 01:48:44,768 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [] 01:48:44,768 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 01:48:44,768 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:48:44,768 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:44,768 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:44,768 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 01:48:44,769 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 01:48:44,769 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-1-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:44,769 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done false 01:48:44,769 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:48:44,769 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:44,769 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:44,769 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: -1, nextIndex: 0 01:48:44,769 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:44,824 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:48:44,824 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:44,824 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 4 01:48:44,824 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:44,826 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:44,826 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 01:48:44,826 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:48:44,826 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:48:44,826 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=null 01:48:44,826 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:48:44,826 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:44,826 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:48:44,829 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:48:44,829 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:48:44,829 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 01:48:44,829 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:44,829 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:48:44,829 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 01:48:44,829 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [] 01:48:44,830 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 01:48:44,830 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:48:44,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:44,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:44,830 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 01:48:44,830 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 01:48:44,831 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 01:48:44,831 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:48:44,831 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:44,831 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:44,831 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:44,831 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 01:48:44,831 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done false 01:48:44,831 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: -1, nextIndex: 0 01:48:44,831 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:44,885 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done true 01:48:44,898 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1305921173] 01:48:44,898 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 01:48:44,898 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1652989729] 01:48:44,899 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1305921173] 01:48:44,900 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:48:44,900 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:44,900 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:44,900 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1603100816] 01:48:44,900 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:48:44,900 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1341379471] 01:48:44,900 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1341379471] 01:48:44,901 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionForwardedToLeaderAfterRetry commit rateLimit adjusted to 40.88563184859886 01:48:44,901 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,901 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1603100816] 01:48:44,901 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 01:48:44,901 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:48:44,901 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1603100816] 01:48:44,901 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:48:44,901 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 01:48:44,901 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Readying 2 components for commit 01:48:44,901 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:48:44,901 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Readying transaction for shard cars 01:48:44,901 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 01:48:44,901 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Readying transaction for shard people 01:48:44,901 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:48:44,901 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 01:48:44,901 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 canCommit 01:48:44,901 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:48:44,902 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars 01:48:44,902 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 01:48:44,902 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:48:44,902 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 = [ 1 ] 01:48:44,902 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 readyTransaction called 01:48:44,902 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 sending 1 batched modifications, ready: true 01:48:44,902 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 01:48:44,902 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:48:44,902 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] for shard people 01:48:44,902 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 01:48:44,902 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 01:48:44,902 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 = [ 1 ] 01:48:44,902 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 readyTransaction called 01:48:44,902 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 sending 1 batched modifications, ready: true 01:48:44,902 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] 01:48:44,903 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, disabling transaction tracking 01:48:44,903 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:48:44,904 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:48:44,904 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, disabling transaction tracking 01:48:44,904 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:48:44,904 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:48:44,904 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:48:44,904 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:44,904 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:44,904 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:48:44,904 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 01:48:44,904 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:44,904 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:44,904 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 01:48:44,904 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,904 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,904 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 of 1 operations, client version 13 01:48:44,904 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,904 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 of 1 operations, client version 13 01:48:44,904 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,904 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 ready 01:48:44,904 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 ready 01:48:44,905 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:48:44,905 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:48:44,905 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 readyTransaction 01:48:44,905 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870)] resolved 01:48:44,905 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:48:44,906 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:48:44,906 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 readyTransaction 01:48:44,906 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1652989729)] resolved 01:48:44,906 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: successfully resolved all cohort actors 01:48:44,906 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 finishCanCommit 01:48:44,906 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870)] 01:48:44,906 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870)] 01:48:44,906 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 01:48:44,906 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:48:44,906 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:48:44,906 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 01:48:44,906 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 01:48:44,906 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:48:44,906 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:48:44,906 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:48:44,906 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 01:48:44,906 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:48:44,906 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 1 01:48:44,906 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 1 01:48:44,906 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:44,906 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:44,907 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:48:44,907 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:48:44,907 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done true 01:48:44,907 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 0, nextIndex: 1 01:48:44,907 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:48:44,907 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 0 01:48:44,907 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 01:48:44,908 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 01:48:44,908 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:48:44,908 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 01:48:44,908 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=4049706} is already disabled 01:48:44,908 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:44,908 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 01:48:44,908 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 0, nextIndex: 1 01:48:44,908 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:48:44,908 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 0 01:48:44,908 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 01:48:44,908 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 01:48:44,908 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:48:44,908 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 01:48:44,908 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=4636425} is already disabled 01:48:44,908 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:44,908 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 01:48:44,908 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,908 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,908 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 validated 01:48:44,908 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:44,908 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: success 01:48:44,909 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 0 01:48:44,909 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:48:44,909 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 0 01:48:44,909 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 01:48:44,909 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 01:48:44,909 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:48:44,909 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 01:48:44,909 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 01:48:44,909 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 0 01:48:44,909 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:48:44,909 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 0 01:48:44,909 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 01:48:44,909 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 01:48:44,909 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:48:44,909 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 01:48:44,909 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 01:48:44,910 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:48:44,910 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1652989729)] 01:48:44,910 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1652989729)] 01:48:44,911 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,911 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,911 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 validated 01:48:44,911 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:44,911 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: success 01:48:44,911 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:48:44,911 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit returning result: true 01:48:44,911 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 commit 01:48:44,911 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 finish commit 01:48:44,911 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870)] 01:48:44,911 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870)] 01:48:44,911 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1652989729)] 01:48:44,911 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1652989729)] 01:48:44,912 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,912 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,912 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 candidate ready 01:48:44,912 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:48:44,912 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,912 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,912 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit - messages: [] 01:48:44,912 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:48:44,912 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 prepared 01:48:44,912 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:44,912 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,912 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 candidate ready 01:48:44,912 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,912 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:48:44,912 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit - messages: [] 01:48:44,912 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:48:44,912 PM [cluster-test-shard-dispatcher-24] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 179 01:48:44,912 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 prepared 01:48:44,912 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:44,912 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,912 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:48:44,912 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,912 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:48:44,913 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:44,913 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:44,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 183 01:48:44,913 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:44,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 01:48:44,913 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 01:48:44,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 01:48:44,913 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 submitted to persistence 01:48:44,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:44,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:44,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:44,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}]]] 01:48:44,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 submitted to persistence 01:48:44,914 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:48:44,914 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 01:48:44,914 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:48:44,914 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:48:44,914 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=179} 01:48:44,914 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 01:48:44,914 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:48:44,914 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 01:48:44,914 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:44,914 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}]]] 01:48:44,914 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:48:44,914 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:48:44,914 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=183} 01:48:44,914 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 01:48:44,914 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:48:44,914 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 01:48:44,914 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:44,914 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 01:48:44,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 01:48:44,915 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 01:48:44,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:48:44,915 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:48:44,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 1 01:48:44,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 01:48:44,915 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 1 01:48:44,915 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 01:48:44,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data Simple{size=179} 01:48:44,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,915 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data Simple{size=183} 01:48:44,915 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:48:44,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:48:44,915 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:48:44,915 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:48:44,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry%23-256835870$e] 01:48:44,915 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-people-testTransactionForwardedToLeaderAfterRetry%231652989729$f] 01:48:44,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-2017029759] with name member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 01:48:44,915 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1361192476] with name member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 01:48:44,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 01:48:44,915 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 01:48:44,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:44,915 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:44,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 01:48:44,915 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 01:48:44,915 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-2017029759] 01:48:44,915 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:48:44,916 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 84.97 μs 01:48:44,916 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-2017029759] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:44,916 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1361192476] 01:48:44,916 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:48:44,916 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 56.40 μs 01:48:44,916 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1361192476] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:44,917 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 1 01:48:44,917 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:48:44,917 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 1 01:48:44,917 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: commit succeeded 01:48:44,917 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 01:48:44,917 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data Simple{size=179} 01:48:44,917 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,917 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 1 01:48:44,917 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:48:44,917 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 1 01:48:44,917 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 01:48:44,917 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data Simple{size=183} 01:48:44,917 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:48:44,917 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-133540784] with name member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 01:48:44,917 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#946403907] with name member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 01:48:44,918 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 01:48:44,918 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 01:48:44,918 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 01:48:44,918 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 01:48:44,918 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#946403907] 01:48:44,918 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:48:44,918 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 129.8 μs 01:48:44,918 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#946403907] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:44,919 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-133540784] 01:48:44,919 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:48:44,919 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 52.43 μs 01:48:44,919 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-133540784] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:44,919 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1603100816] 01:48:44,920 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:48:44,920 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1341379471] 01:48:44,920 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1341379471] 01:48:44,920 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1603100816] 01:48:44,920 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 01:48:44,920 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#1269956772] 01:48:44,921 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#1269956772] 01:48:44,921 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:48:44,921 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:44,921 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:48:44,921 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 01:48:44,921 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:48:44,921 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] for shard people, trying to use DirectTransactionContextWrapper 01:48:44,921 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:48:44,921 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 01:48:44,921 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 01:48:44,921 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Readying 2 components for commit 01:48:44,921 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Readying transaction for shard cars 01:48:44,921 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 = [ 1 ] 01:48:44,921 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 readyTransaction called 01:48:44,921 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 sending 1 batched modifications, ready: true 01:48:44,921 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 01:48:44,922 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Readying transaction for shard people 01:48:44,922 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 = [ 1 ] 01:48:44,922 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 readyTransaction called 01:48:44,922 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 sending 1 batched modifications, ready: true 01:48:44,922 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] 01:48:44,922 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 canCommit 01:48:44,923 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:48:44,923 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 of 1 operations, client version 13 01:48:44,923 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:48:44,923 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 ready 01:48:44,923 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:48:44,923 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 of 1 operations, client version 13 01:48:44,923 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:48:44,923 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 ready 01:48:44,924 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 readyTransaction 01:48:44,924 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870)] resolved 01:48:44,924 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 modifications completed with ReadyTransactionReply [version=13] 01:48:44,924 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 readyTransaction 01:48:44,924 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1652989729)] resolved 01:48:44,924 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: successfully resolved all cohort actors 01:48:44,924 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 modifications completed with ReadyTransactionReply [version=13] 01:48:44,924 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 finishCanCommit 01:48:44,924 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870)] 01:48:44,924 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870)] 01:48:44,925 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:48:44,925 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:48:44,925 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 validated 01:48:44,925 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:44,925 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: success 01:48:44,925 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:48:44,925 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1652989729)] 01:48:44,925 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1652989729)] 01:48:44,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:48:44,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:48:44,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 validated 01:48:44,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:44,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: success 01:48:44,927 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:48:44,927 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: canCommit returning result: true 01:48:44,927 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:48:44,927 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:44,927 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:48:44,927 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 01:48:44,927 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}] 01:48:44,927 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] for shard people, trying to use DirectTransactionContextWrapper 01:48:44,927 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:48:44,927 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 01:48:44,927 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person 01:48:44,927 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Readying 2 components for commit 01:48:44,927 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Readying transaction for shard cars 01:48:44,927 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 = [ 1 ] 01:48:44,928 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 readyTransaction called 01:48:44,928 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 sending 1 batched modifications, ready: true 01:48:44,928 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 01:48:44,928 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Readying transaction for shard people 01:48:44,928 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 = [ 1 ] 01:48:44,928 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 readyTransaction called 01:48:44,928 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 sending 1 batched modifications, ready: true 01:48:44,928 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] 01:48:44,928 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1305921173] 01:48:44,928 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:48:44,929 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:44,929 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:44,929 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:48:44,929 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 of 1 operations, client version 13 01:48:44,929 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:48:44,929 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 ready 01:48:44,929 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:48:44,929 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 of 1 operations, client version 13 01:48:44,929 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:48:44,929 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 ready 01:48:44,930 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 readyTransaction 01:48:44,930 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 modifications completed with ReadyTransactionReply [version=13] 01:48:44,930 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 readyTransaction 01:48:44,930 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 modifications completed with ReadyTransactionReply [version=13] 01:48:44,980 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:45,30 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:45,80 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:45,126 PM [cluster-test-shard-dispatcher-22] [WARN] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 205, lastApplied : 1, commitIndex : 1 01:48:45,131 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:45,181 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:45,231 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:45,282 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:45,332 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:45,382 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:45,433 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:45,483 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:45,533 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:45,584 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:45,634 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:45,644 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:48:45,685 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:45,735 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:45,786 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:45,836 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:45,886 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:45,937 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:45,988 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:46,39 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:46,89 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:46,139 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:46,190 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:46,240 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:46,290 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:46,341 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:46,391 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:46,441 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:46,492 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:46,542 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:46,593 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:46,643 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:46,694 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:46,744 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:46,794 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:46,845 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:46,895 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:46,945 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:46,996 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:47,46 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:47,97 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:47,147 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:47,197 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:47,248 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:47,298 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:47,349 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:47,399 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:47,449 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:47,500 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:47,550 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:47,601 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:47,651 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:47,701 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:47,752 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:47,802 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:47,853 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:47,903 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:47,953 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:48,4 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:48,54 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:48,104 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:48,155 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:48,205 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:48,255 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:48,306 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:48,356 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:48,406 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:48,457 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:48,507 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:48,557 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:48,608 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:48,658 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:48,708 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:48,759 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:48,809 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:48,859 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:48,910 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:48,960 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:49,10 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:49,61 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:49,111 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:49,162 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:49,212 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:49,262 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:49,313 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:49,363 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:49,398 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 closeTransaction called 01:48:49,398 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-3:datastore-testLeadershipTransferOnShutdown@0:0_14#-5413732)] [ERROR] [03/11/2024 13:48:49.400] [com.google.common.base.internal.Finalizer] [akka://cluster-test@127.0.0.1:2559/] swallowing exception during message send java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.remote.artery.Association.setupStopQuarantinedTimer(Association.scala:633) at akka.remote.artery.Association.send(Association.scala:399) at akka.remote.artery.ArteryTransport.send(ArteryTransport.scala:707) at akka.remote.RemoteActorRef.$bang(RemoteActorRefProvider.scala:737) at akka.actor.ActorRef.tell(ActorRef.scala:131) at akka.actor.ActorSelection$.rec$1(ActorSelection.scala:287) at akka.actor.ActorSelection$.deliverSelection(ActorSelection.scala:291) at akka.actor.ActorSelection.tell(ActorSelection.scala:50) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.sendOperationAsync(ActorUtils.java:398) at org.opendaylight.controller.cluster.datastore.RemoteTransactionContext.closeTransaction(RemoteTransactionContext.java:84) at org.opendaylight.controller.cluster.datastore.TransactionContextCleanup.finalizeReferent(TransactionContextCleanup.java:60) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at com.google.common.base.internal.Finalizer.finalizeReference(Finalizer.java:208) at com.google.common.base.internal.Finalizer.cleanUp(Finalizer.java:168) at com.google.common.base.internal.Finalizer.run(Finalizer.java:146) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:48:49,413 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:49,464 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:49,514 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:49,565 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:49,615 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:49,665 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:49,716 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:49,766 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:49,816 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:49,867 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:49,917 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:49,967 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:50,18 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:50,68 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:50,119 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:50,169 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:50,219 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:50,270 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:50,320 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:50,371 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:50,421 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:50,471 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:50,521 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:50,572 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:50,622 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:50,672 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:50,723 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:50,773 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:50,823 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:50,873 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:50,924 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] 01:48:50,974 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:48:50,975 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:50,975 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:50,975 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry - waiting on 2 shards 01:48:50,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onShutDown 01:48:50,975 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onShutDown 01:48:50,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Initiating leader transfer 01:48:50,975 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Initiating leader transfer 01:48:50,975 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=13] 01:48:50,975 PM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:50,975 PM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:50,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7a213873 01:48:50,975 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@50243c41 01:48:50,975 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=13] 01:48:50,975 PM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:50,975 PM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:50,981 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry] 01:48:50,981 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry] 01:48:50,984 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=13] 01:48:50,984 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:50,984 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:50,984 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=13] 01:48:50,984 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:50,984 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:51,194 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Attempting to transfer leadership 01:48:51,194 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Attempting to transfer leadership 01:48:51,194 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:51,194 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:51,196 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, matchIndex: 1, lastIndex: 1, isVoting: true 01:48:51,196 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, matchIndex: 1, lastIndex: 1, isVoting: true 01:48:51,196 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Follower's log matches - sending ElectionTimeout 01:48:51,196 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Follower's log matches - sending ElectionTimeout 01:48:51,196 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:51,196 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:51,197 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Leader transfer complete 01:48:51,197 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Leader transfer complete 01:48:51,394 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All shards shutdown - sending PoisonPill to self 01:48:51,394 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:48:51,394 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:51,394 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-489: Closing 01:48:51,395 PM [cluster-test-shard-dispatcher-34] [INFO] TestShard - Stopping Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:51,395 PM [cluster-test-shard-dispatcher-34] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-488: Closing 01:48:51,395 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Aborting 2 pending queued transactions 01:48:51,395 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Aborting 2 pending queued transactions 01:48:51,395 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 01:48:51,396 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 01:48:51,396 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:48:51,396 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 01:48:51,398 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 01:48:51,399 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [akka.actor.Status$Failure] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-256835870] to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry%23-256835870$i] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry%23-256835870$i] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:51,402 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:51,402 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:51,405 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:51,405 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:51,405 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:51,406 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:51,410 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:48:51,415 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 01:48:51,416 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:48:51,416 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:48:51,416 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:48:51,417 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:48:51,417 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 01:48:51,417 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 01:48:51,417 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 01:48:51,418 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:51,418 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:52,421 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4437841914933822848] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:52,430 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-5251907920139704957] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:52,436 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:52,436 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:52,438 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:52.439] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:52,456 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:52,456 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:52,460 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:52.461] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:52,486 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:52,492 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-5117018373571484816] 01:48:52,492 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:48:52,494 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:48:52,494 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:48:52,499 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:52,499 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:52,505 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:48:52,505 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:48:52,505 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:48:52,505 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:48:52,511 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [6812341444201241480] 01:48:52,512 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:48:52,514 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:52,514 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:48:52,519 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:52,519 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:52,526 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:48:52,534 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2808616133907044169] 01:48:52,534 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:48:52,536 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:52,536 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:48:52,541 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:52,541 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:52,546 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:52,547 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:52,548 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,563 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,563 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1912612764] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:52,568 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 01:48:52,568 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:48:52,568 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@19072366 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4f15ee91 01:48:52,570 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@37546c86 01:48:52,570 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1710164932570, None)) 01:48:52,572 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164932569,0,0)) 01:48:52,572 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4f15ee91 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@73f7a082 01:48:52,572 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 01:48:52,572 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:52,574 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,574 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:52,574 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,574 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:52,574 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,574 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@57c0e544 01:48:52,574 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:52,574 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,576 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,576 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,576 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,576 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1966075188] 01:48:52,576 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 01:48:52,576 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1966075188] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:52,577 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 01:48:52,579 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,579 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1923540563] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:52,580 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,580 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,580 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 01:48:52,580 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:48:52,580 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6995c9ca to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@753bcb9b 01:48:52,581 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@710441e4 01:48:52,581 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1710164932580, None)) 01:48:52,581 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164932579,0,0)) 01:48:52,581 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5cfa01b 01:48:52,581 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@753bcb9b to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5789460 01:48:52,581 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 01:48:52,581 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:52,581 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,582 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 01:48:52,582 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#85226054] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:52,582 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-2:snapshot-read#845715042] 01:48:52,582 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#804706589 created and ready for shard:member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,582 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-2:snapshot-read#845715042] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:52,582 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#804706589] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:52,582 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 01:48:52,583 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:52,583 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:52,583 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 01:48:52,583 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:48:52,583 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 01:48:52,583 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 01:48:52,583 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 01:48:52,583 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,583 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 01:48:52,584 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#1462180889 created and ready for shard:member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,584 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-1:snapshot-read#1219849317] 01:48:52,584 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#1462180889] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:52,584 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-1:snapshot-read#1219849317] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:52,584 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:52,584 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:52,584 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 01:48:52,584 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 01:48:52,584 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 01:48:52,584 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:48:52,584 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 01:48:52,585 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,585 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 01:48:52,590 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:48:52,591 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:48:52,592 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:52,600 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:52,600 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:52,600 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,600 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,632 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1912612764] 01:48:52,632 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: findLocalShard : cars 01:48:52,632 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1966075188] 01:48:52,874 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:48:52,874 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Received ElectionTimeout - switching to Candidate 01:48:52,874 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 01:48:52,874 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:52,874 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 1 01:48:52,875 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:52,875 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 01:48:52,875 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 01:48:52,875 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:48:52,875 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Follower to Candidate 01:48:52,875 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Follower to Candidate 01:48:53,115 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 01:48:53,115 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,115 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 2 01:48:53,164 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:48:53,394 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 01:48:53,394 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,394 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 3 01:48:53,516 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:48:53,516 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:48:53,517 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:53,517 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,517 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,517 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:53,517 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,517 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,517 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,517 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,524 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:53,524 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,524 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,524 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:53,524 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,524 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,524 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,555 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:48:53,624 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 01:48:53,624 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,624 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 4 01:48:53,625 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,625 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,626 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:53,626 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1966075188] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:53,626 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Cluster state: [] 01:48:53,626 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:48:53,626 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:48:53,626 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=null 01:48:53,627 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:48:53,627 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,627 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:48:53,628 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:48:53,628 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:48:53,628 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Election: Leader has following peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 01:48:53,628 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:53,628 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:53,628 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:48:53,628 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 01:48:53,629 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: became leader with frontend state for [] 01:48:53,629 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Candidate to Leader 01:48:53,629 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 01:48:53,629 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:53,629 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:53,629 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Candidate to Leader 01:48:53,629 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 01:48:53,629 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1966075188] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:53,629 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:48:53,629 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,630 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 01:48:53,630 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 01:48:53,630 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:53,630 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:53,630 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 01:48:53,630 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: -1, nextIndex: 0 01:48:53,631 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:53,638 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:53,641 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:53,643 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,648 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,648 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-593646517] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:53,650 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 01:48:53,650 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:48:53,650 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@44ab99c4 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5e063ae6 01:48:53,650 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@296965f8 01:48:53,650 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1710164933650, None)) 01:48:53,650 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,650 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,651 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:53,651 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,651 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:53,651 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,651 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:53,651 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,651 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164933649,0,0)) 01:48:53,651 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5e063ae6 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@610d751f 01:48:53,651 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5095238d 01:48:53,651 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 01:48:53,651 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:53,651 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,652 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 01:48:53,652 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1923540563] 01:48:53,652 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,652 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,652 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#922312220] 01:48:53,652 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 Queuing TransactionOperation 01:48:53,652 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,652 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:48:53,652 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,652 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#922312220] 01:48:53,652 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,652 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,652 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#922312220] 01:48:53,652 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, primaryVersion=13] 01:48:53,652 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, primaryVersion=13] 01:48:53,652 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] for shard cars 01:48:53,652 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] found - trying create transaction 01:48:53,652 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 01:48:53,652 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] 01:48:53,652 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 01:48:53,653 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#922312220] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:53,653 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-3:snapshot-read#-1438235519] 01:48:53,653 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#1101932585] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:53,653 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#1101932585 created and ready for shard:member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,655 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:53,655 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:53,655 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 01:48:53,655 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:48:53,655 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 01:48:53,655 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,655 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,655 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 01:48:53,655 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 01:48:53,656 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,656 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 01:48:53,658 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, disabling transaction tracking 01:48:53,658 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 01:48:53,658 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 01:48:53,658 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-3:snapshot-read#-1438235519] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:53,658 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:48:53,658 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:53,658 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:53,658 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,658 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]]] 01:48:53,658 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating transaction : member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 01:48:53,659 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_15#-1149360644] 01:48:53,659 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_15#-1149360644] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:53,661 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_15#-1149360644, transactionId=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0, version=13] 01:48:53,661 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:48:53,661 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_15#-1149360644)] 01:48:53,661 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]]] 01:48:53,661 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 01:48:53,661 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:48:53,661 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 01:48:53,661 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 01:48:53,661 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 1 01:48:53,661 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:53,664 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 01:48:53,664 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=null 01:48:53,664 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:48:53,664 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,665 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 01:48:53,665 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done true 01:48:53,666 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 ReadData operation succeeded 01:48:53,666 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:48:53,668 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 01:48:53,668 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:53,668 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:53,668 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 01:48:53,668 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 01:48:53,672 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,672 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader)_slicer-id-494: Closing 01:48:53,672 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 01:48:53,672 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,673 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:53,673 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:53,678 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:53,678 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:53,678 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:53,678 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:53,680 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:48:53,696 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,696 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 01:48:53,697 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,697 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 01:48:53,697 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:53,697 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 01:48:53,697 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : true 01:48:53,697 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:48:53,699 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionRetryWithInitialAskTimeoutExOnCreateTx commit rateLimit adjusted to 40.88563184859886 01:48:53,699 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:48:53,699 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:53,699 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:48:53,699 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] found - trying create transaction 01:48:53,699 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] 01:48:53,699 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Queuing TransactionOperation 01:48:53,699 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Readying 1 components for commit 01:48:53,699 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Readying transaction for shard cars 01:48:53,699 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Queuing TransactionOperation 01:48:53,699 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 canCommit 01:48:54,215 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)]] after [500 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 01:48:54,224 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 01:48:54,224 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 01:48:54,224 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 01:48:54,224 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Received ElectionTimeout - switching to Candidate 01:48:54,224 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 01:48:54,224 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:54,224 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 5 01:48:54,224 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=4] to peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:54,225 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:54,225 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:48:54,225 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 01:48:54,225 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 01:48:54,225 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:48:54,225 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:48:54,225 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Follower to Candidate 01:48:54,225 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=null, leaderPayloadVersion=-1] 01:48:54,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:54,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:54,225 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Follower to Candidate 01:48:54,227 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:54,227 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Cluster state: [] 01:48:54,227 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:48:54,227 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 4 - updating term 01:48:54,227 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=null 01:48:54,227 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 01:48:54,227 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:54,227 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:48:54,228 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:48:54,228 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:48:54,228 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 01:48:54,234 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Election: Leader has following peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 01:48:54,234 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:54,234 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:54,236 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 5 01:48:54,236 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 01:48:54,236 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: became leader with frontend state for [] 01:48:54,236 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:48:54,236 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:48:54,236 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Replicate message: identifier: null, logIndex: 1, payload: class org.opendaylight.controller.cluster.raft.persisted.NoopPayload, isSendImmediate: true 01:48:54,236 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:54,236 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:54,237 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:48:54,237 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:54,237 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Candidate to PreLeader 01:48:54,237 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 01:48:54,237 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 01:48:54,237 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:54,237 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:54,237 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Candidate to PreLeader 01:48:54,238 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:48:54,238 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 01:48:54,238 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:54,238 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:54,238 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 01:48:54,239 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: -1, nextIndex: 0 01:48:54,239 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:54,239 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:54,239 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}], SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c]]] 01:48:54,240 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done true 01:48:54,240 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}], SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c]]] 01:48:54,240 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 2 01:48:54,240 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:48:54,240 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 01:48:54,240 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 01:48:54,240 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:54,240 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c 01:48:54,240 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:48:54,240 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 2 01:48:54,241 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=1, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:54,241 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 01:48:54,241 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:48:54,241 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 1, nextIndex: 2 01:48:54,241 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 4 does not match the current term 5 01:48:54,241 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 01:48:54,241 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Setting last applied to 0 01:48:54,241 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from -1 to 0 01:48:54,241 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 01:48:54,242 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:48:54,242 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} does not match any client, pre-disabling client. 01:48:54,242 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: leader state for ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} not found 01:48:54,242 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Setting last applied to 1 01:48:54,242 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from 0 to 1 01:48:54,242 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c 01:48:54,242 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:54,242 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Received ApplyState [identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, replicatedLogEntry=SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]] - lastApplied: 1, lastIndex: 1 01:48:54,242 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Election: Leader has following peers: [member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 01:48:54,242 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:54,242 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:54,242 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 5 01:48:54,242 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader)_slicer-id-497: Closing 01:48:54,242 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 01:48:54,242 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=1 01:48:54,242 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from PreLeader to Leader 01:48:54,242 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from PreLeader to Leader 01:48:54,242 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 01:48:54,243 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Commit index set to 1 01:48:54,243 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 01:48:54,243 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Setting last applied to 0 01:48:54,243 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from -1 to 0 01:48:54,243 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 01:48:54,243 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:48:54,243 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} does not match any client, pre-disabling client. 01:48:54,243 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Setting last applied to 1 01:48:54,243 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from 0 to 1 01:48:54,243 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c 01:48:54,243 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=1 01:48:54,246 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 01:48:54,246 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1923540563] 01:48:54,246 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:48:54,246 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#85226054, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@27d75a62] 01:48:54,246 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#85226054, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@27d75a62] 01:48:54,246 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#85226054)] found - trying create transaction 01:48:54,246 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#85226054)] 01:48:54,246 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating transaction : member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:48:54,247 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:1_16#400243914] 01:48:54,247 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:1_16#400243914, transactionId=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0, version=13] 01:48:54,247 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:48:54,247 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 directCommit called 01:48:54,247 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 sending 1 batched modifications, ready: true 01:48:54,247 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:1_16#400243914)] 01:48:54,247 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:1_16#400243914] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:54,247 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:48:54,247 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: handleForwardedReadyTransaction for member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:48:54,247 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Readying transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0, client version 13 01:48:54,247 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: readying transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:48:54,247 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 ready 01:48:54,247 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Validating transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:48:54,247 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 validated 01:48:54,247 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:54,247 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: canCommit for member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: success 01:48:54,247 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Committing transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:48:54,247 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Preparing transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:48:54,247 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 candidate ready 01:48:54,247 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: canCommit - candidate rootPath: / 01:48:54,248 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: canCommit - messages: [] 01:48:54,248 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 01:48:54,248 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 prepared 01:48:54,248 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:54,248 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Finishing commit for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:48:54,248 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Starting commit for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:48:54,248 PM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 191 01:48:54,248 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}] 01:48:54,248 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}] 01:48:54,248 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Replicate message: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:54,248 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:48:54,248 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:48:54,248 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:54,248 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=1, prevLogTerm=5, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}]]] 01:48:54,248 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 submitted to persistence 01:48:54,249 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}] 01:48:54,250 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=1, prevLogTerm=5, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}]]] 01:48:54,250 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 01:48:54,250 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:48:54,250 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log Simple{size=191} 01:48:54,250 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}] 01:48:54,250 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 3 01:48:54,250 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:54,250 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}] 01:48:54,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 2, nextIndex: 3 01:48:54,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:48:54,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Setting last applied to 2 01:48:54,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from 1 to 2 01:48:54,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 2 data Simple{size=191} 01:48:54,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Resuming commit of transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:48:54,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: commit - successfulFromPrevious: [] 01:48:54,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 01:48:54,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_shard-cars-member-2%3Adatastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx%400%3A1_16%23400243914$c] 01:48:54,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#391977572] with name member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher 01:48:54,251 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 successfully completed direct commit 01:48:54,251 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 modifications completed with CommitTransactionReply [version=13] 01:48:54,251 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:48:54,251 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#391977572] 01:48:54,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:48:54,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=2 01:48:54,251 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: publishChanges: rootPath: / 01:48:54,251 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Issuing gracefulStop to shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:54,251 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx - waiting on 1 shards 01:48:54,251 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onShutDown 01:48:54,252 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Elapsed time for generation of change events for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher: 107.0 μs 01:48:54,252 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#391977572] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:54,252 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:54,252 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 01:48:54,252 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 01:48:54,252 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All shards shutdown - sending PoisonPill to self 01:48:54,252 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:54,253 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#85226054] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:54,253 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:48:54,253 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:48:54,253 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: shutting down 01:48:54,254 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:48:54,254 PM [main] [WARN] ActorUtils - ShardManager for testTransactionRetryWithInitialAskTimeoutExOnCreateTx data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:48:54,254 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:48:54,254 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#1771623576] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1771623576] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:54,254 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-593646517] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-593646517] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:54,254 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-593646517] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-593646517] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:54,254 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:48:54,254 PM [main] [WARN] ActorUtils - ShardManager for testTransactionRetryWithInitialAskTimeoutExOnCreateTx data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:48:54,255 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:48:54,255 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:48:54,255 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:54,255 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader)_slicer-id-498: Closing 01:48:54,255 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 01:48:54,255 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:48:54,256 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:54,256 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:54,695 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-5117018373571484816] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:55,276 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [6812341444201241480] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:55,276 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:55,277 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:55,278 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:55.280] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config [WARN] [03/11/2024 13:48:55.300] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:55,338 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:48:55,348 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4356589091775288550] 01:48:55,348 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:48:55,349 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:48:55,349 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:48:55,356 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:55,356 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:55,360 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:48:55,360 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:48:55,360 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:48:55,368 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:55,373 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-671284327394077265] 01:48:55,373 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:48:55,374 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:55,374 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:48:55,381 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:55,381 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:55,388 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:48:55,396 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [7266309612804381080] 01:48:55,396 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:48:55,398 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:55,398 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:48:55,398 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:55,398 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:55,406 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:55,410 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:55,412 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithMultipleShards 01:48:55,421 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 01:48:55,422 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-832803499] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:55,424 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: no initial generation override, starting from 0 01:48:55,424 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:48:55,424 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@62c5ca22 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@69fb497a 01:48:55,424 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6dc4d09d 01:48:55,424 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithMultipleShards 01:48:55,424 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:55,424 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:48:55,425 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:55,425 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:55,425 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:48:55,425 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, 0, 1710164935425, None)) 01:48:55,425 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164935424,0,0)) 01:48:55,425 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@69fb497a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@19244ad0 01:48:55,425 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} started 01:48:55,425 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5b568ca5 01:48:55,426 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:55,427 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:55,429 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithMultipleShards 01:48:55,429 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:55,429 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:55,430 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:55,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:55,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards on shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#2028196473] 01:48:55,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:48:55,430 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:48:55,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:48:55,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards on shard member-1-shard-people-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-729046024] 01:48:55,430 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-cars-testReadWriteTransactionWithMultipleShards, persistent : true 01:48:55,430 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#2028196473] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:55,431 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#1651686063 created and ready for shard:member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:55,431 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#1651686063] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:55,431 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadWriteTransactionWithMultipleShards with journal batch size 1 01:48:55,431 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-341165072] 01:48:55,431 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-341165072] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:55,431 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-people-testReadWriteTransactionWithMultipleShards, persistent : true 01:48:55,431 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-notifier#51130095 created and ready for shard:member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:48:55,431 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-729046024] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:55,431 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1185012511] 01:48:55,432 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-people-testReadWriteTransactionWithMultipleShards with journal batch size 1 01:48:55,433 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-notifier#51130095] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:55,433 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1185012511] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:55,436 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-468789284] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:55,436 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:55,436 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:55,436 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [people] 01:48:55,436 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is initialized 01:48:55,436 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:55,436 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:48:55,436 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:55,436 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 01:48:55,436 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:48:55,436 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 01:48:55,436 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [cars] 01:48:55,436 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is initialized 01:48:55,437 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 01:48:55,437 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 01:48:55,437 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 01:48:55,437 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 01:48:55,437 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from null to Follower 01:48:55,437 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from null to Follower 01:48:55,442 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:55,446 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 01:48:55,446 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithMultipleShards 01:48:55,446 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:55,446 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:48:55,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: no initial generation override, starting from 0 01:48:55,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:48:55,447 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@22d218ba to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@41cc764f 01:48:55,447 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@45f3faae 01:48:55,447 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, 0, 1710164935447, None)) 01:48:55,447 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164935446,0,0)) 01:48:55,447 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@41cc764f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3578911f 01:48:55,447 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} started 01:48:55,447 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7e4b997f 01:48:55,447 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:55,447 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:55,448 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:48:55,448 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Shard created : member-2-shard-cars-testReadWriteTransactionWithMultipleShards, persistent : true 01:48:55,448 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadWriteTransactionWithMultipleShards with journal batch size 1 01:48:55,448 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Shard created : member-2-shard-people-testReadWriteTransactionWithMultipleShards, persistent : true 01:48:55,449 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Starting recovery for member-2-shard-people-testReadWriteTransactionWithMultipleShards with journal batch size 1 01:48:55,449 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards#-2129187782] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:55,449 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards#-944492632] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:55,449 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-379378049] 01:48:55,449 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#-1490383606 created and ready for shard:member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:55,449 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-notifier#2134074614 created and ready for shard:member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:48:55,449 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#1609408252] 01:48:55,449 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#-1490383606] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:55,449 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-379378049] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:55,449 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-notifier#2134074614] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:55,449 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#1609408252] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:55,451 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:55,451 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:55,451 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:48:55,451 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 01:48:55,451 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [people] 01:48:55,451 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards is initialized 01:48:55,451 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:55,451 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:55,452 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:48:55,452 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 01:48:55,452 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 01:48:55,452 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 01:48:55,452 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 01:48:55,452 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [cars] 01:48:55,452 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards is initialized 01:48:55,452 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-2-shard-people-testReadWriteTransactionWithMultipleShards from null to Follower 01:48:55,452 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 01:48:55,452 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-2-shard-cars-testReadWriteTransactionWithMultipleShards from null to Follower 01:48:55,453 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:55,455 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:48:55,455 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:48:55,455 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:55,455 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:55,455 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:55,455 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:48:55,455 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:48:55,455 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:48:55,455 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:55,497 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-832803499] 01:48:55,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: findLocalShard : cars 01:48:55,498 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#2028196473] 01:48:55,700 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:48:55,700 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:48:55,700 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testReadWriteTransactionWithMultipleShards] 01:48:55,700 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:48:55,700 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 01:48:55,700 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:55,700 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Closing 01:48:55,701 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:48:55,701 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 01:48:55,701 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from Follower to Candidate 01:48:55,701 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:48:55,701 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:48:55,701 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadWriteTransactionWithMultipleShards] 01:48:55,701 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:55,701 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 01:48:55,701 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from Follower to Candidate 01:48:55,701 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:55,701 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Closing 01:48:55,701 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:48:55,701 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 01:48:55,701 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from Follower to Candidate 01:48:55,701 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from Follower to Candidate 01:48:55,951 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:48:55,951 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:55,951 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 01:48:56,20 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:48:56,20 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:48:56,20 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 01:48:56,42 PM [cluster-test-shard-dispatcher-31] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:48:56,52 PM [cluster-test-shard-dispatcher-31] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:48:56,211 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:48:56,211 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:56,211 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 01:48:56,250 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:48:56,250 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:48:56,250 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 01:48:56,371 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:48:56,371 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:48:56,371 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:56,371 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:56,371 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:56,371 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards 01:48:56,371 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:48:56,372 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:56,372 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:56,372 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:56,372 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:48:56,372 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:48:56,372 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:56,372 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:48:56,376 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:48:56,376 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:56,376 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:56,376 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards 01:48:56,376 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:48:56,376 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:48:56,376 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:56,376 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:56,376 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:48:56,376 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:48:56,417 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:48:56,500 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:48:56,500 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:56,500 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 01:48:56,500 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:56,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:56,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Cluster state: [] 01:48:56,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:48:56,511 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:48:56,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 01:48:56,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:48:56,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:56,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:48:56,514 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:48:56,514 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:48:56,514 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadWriteTransactionWithMultipleShards] 01:48:56,514 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:56,514 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:48:56,514 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:48:56,514 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: became leader with frontend state for [] 01:48:56,514 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from Candidate to Leader 01:48:56,514 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:48:56,514 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:56,514 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:56,515 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from Candidate to Leader 01:48:56,515 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Closing 01:48:56,515 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testReadWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:56,516 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:48:56,516 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:56,516 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:56,516 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithMultipleShards status sync done false 01:48:56,516 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:48:56,516 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:56,570 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:48:56,570 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:48:56,570 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 01:48:56,570 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:48:56,572 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:48:56,572 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Cluster state: [] 01:48:56,572 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:48:56,572 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:48:56,572 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 01:48:56,572 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:48:56,572 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:48:56,572 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:48:56,573 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:48:56,573 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:48:56,573 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testReadWriteTransactionWithMultipleShards] 01:48:56,573 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:56,574 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:48:56,574 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:48:56,574 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: became leader with frontend state for [] 01:48:56,574 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from Candidate to Leader 01:48:56,574 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:48:56,574 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:56,574 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:56,574 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from Candidate to Leader 01:48:56,574 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All Shards are ready - data store testReadWriteTransactionWithMultipleShards is ready 01:48:56,575 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Closing 01:48:56,575 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testReadWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:48:56,575 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:48:56,575 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:56,575 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:56,575 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All Shards are ready - data store testReadWriteTransactionWithMultipleShards is ready 01:48:56,575 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testReadWriteTransactionWithMultipleShards status sync done false 01:48:56,576 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:48:56,576 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:48:56,606 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-832803499] 01:48:56,606 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: findLocalShard : people 01:48:56,606 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-729046024] 01:48:56,606 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testReadWriteTransactionWithMultipleShards commit rateLimit adjusted to 40.88563184859886 01:48:56,607 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,607 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-468789284] 01:48:56,607 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:48:56,607 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 01:48:56,607 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-468789284] 01:48:56,607 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 01:48:56,607 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 01:48:56,607 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying 2 components for commit 01:48:56,607 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:48:56,607 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying transaction for shard cars 01:48:56,607 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 01:48:56,607 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 01:48:56,607 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying transaction for shard people 01:48:56,607 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 01:48:56,607 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 01:48:56,607 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 canCommit 01:48:56,607 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 01:48:56,607 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] for shard cars 01:48:56,607 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] found - trying create transaction 01:48:56,607 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] for shard people 01:48:56,607 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] 01:48:56,607 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] found - trying create transaction 01:48:56,607 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] 01:48:56,609 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, disabling transaction tracking 01:48:56,609 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, disabling transaction tracking 01:48:56,609 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 01:48:56,609 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 01:48:56,609 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 01:48:56,609 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 01:48:56,609 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:48:56,609 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:56,609 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:48:56,609 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:56,610 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}]]] 01:48:56,610 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,610 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:56,610 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:48:56,610 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}]]] 01:48:56,610 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,610 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_17#248774404] 01:48:56,610 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_17#248774404] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:56,610 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_18#1132724352] 01:48:56,611 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_18#1132724352] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:56,611 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 01:48:56,611 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 01:48:56,612 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}]]] 01:48:56,612 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:48:56,612 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:48:56,612 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 01:48:56,612 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 01:48:56,612 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 1 01:48:56,612 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:56,612 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}]]] 01:48:56,612 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:48:56,612 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:48:56,612 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 01:48:56,612 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 01:48:56,612 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 1 01:48:56,612 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:56,613 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_17#248774404, transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] 01:48:56,613 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:48:56,613 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_18#1132724352, transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] 01:48:56,613 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 = [ 1 ] 01:48:56,613 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction called 01:48:56,613 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 sending 1 batched modifications, ready: true 01:48:56,613 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_17#248774404)] 01:48:56,613 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 01:48:56,613 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 01:48:56,613 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithMultipleShards status sync done true 01:48:56,613 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 = [ 1 ] 01:48:56,613 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction called 01:48:56,613 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 sending 1 batched modifications, ready: true 01:48:56,613 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 01:48:56,613 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_18#1132724352)] 01:48:56,613 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testReadWriteTransactionWithMultipleShards status sync done true 01:48:56,614 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:48:56,614 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:48:56,614 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:48:56,614 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:48:56,614 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 01:48:56,614 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:48:56,614 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 01:48:56,614 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:48:56,614 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 01:48:56,614 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:48:56,614 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 01:48:56,614 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-1-shard-people-testReadWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:48:56,614 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:48:56,614 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, nanosAgo=5004932} is already disabled 01:48:56,614 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:48:56,614 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, nanosAgo=5145704} is already disabled 01:48:56,614 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:56,614 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:48:56,614 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:48:56,615 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:48:56,615 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,615 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: handleForwardedReadyTransaction for member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,615 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, client version 13 01:48:56,615 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,615 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 ready 01:48:56,615 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,615 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: handleForwardedReadyTransaction for member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,615 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, client version 13 01:48:56,615 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,615 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 ready 01:48:56,616 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 0 01:48:56,616 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:48:56,616 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 01:48:56,616 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:48:56,616 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 01:48:56,616 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:48:56,616 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendMetadata - member-2-shard-people-testReadWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:48:56,616 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:48:56,616 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 0 01:48:56,616 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:48:56,616 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 01:48:56,616 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:48:56,616 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 01:48:56,616 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:48:56,616 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:48:56,616 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:48:56,616 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction 01:48:56,616 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:48:56,616 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction 01:48:56,617 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#2028196473)] resolved 01:48:56,617 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:48:56,617 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-729046024)] resolved 01:48:56,617 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: successfully resolved all cohort actors 01:48:56,617 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 finishCanCommit 01:48:56,617 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#2028196473)] 01:48:56,617 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#2028196473)] 01:48:56,617 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Can committing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,617 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,617 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 validated 01:48:56,617 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:56,617 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: canCommit for member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: success 01:48:56,618 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:48:56,618 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-729046024)] 01:48:56,618 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-729046024)] 01:48:56,619 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Can committing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,619 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,619 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 validated 01:48:56,619 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:48:56,619 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: canCommit for member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: success 01:48:56,620 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:48:56,620 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit returning result: true 01:48:56,620 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 commit 01:48:56,620 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 finish commit 01:48:56,620 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#2028196473)] 01:48:56,620 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#2028196473)] 01:48:56,620 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-729046024)] 01:48:56,620 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-729046024)] 01:48:56,620 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Committing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,620 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,620 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Committing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,620 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 candidate ready 01:48:56,620 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:48:56,620 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,621 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - messages: [] 01:48:56,621 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 candidate ready 01:48:56,621 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:48:56,621 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 prepared 01:48:56,621 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:48:56,621 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:56,621 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Finishing commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,621 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,621 PM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 179 01:48:56,621 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - messages: [] 01:48:56,621 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:48:56,621 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:48:56,621 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 prepared 01:48:56,621 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:48:56,621 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:56,621 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:56,621 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:56,621 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 01:48:56,621 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 submitted to persistence 01:48:56,621 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:48:56,621 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:48:56,621 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Finishing commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,621 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,622 PM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 183 01:48:56,622 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 01:48:56,622 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 01:48:56,622 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:48:56,622 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:48:56,622 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:48:56,622 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}]]] 01:48:56,622 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 submitted to persistence 01:48:56,622 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 01:48:56,622 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 01:48:56,622 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:48:56,622 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:48:56,623 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=179} 01:48:56,623 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:48:56,623 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 2 01:48:56,623 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:56,623 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}]]] 01:48:56,623 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:48:56,623 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:48:56,623 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=183} 01:48:56,623 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 01:48:56,623 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 2 01:48:56,623 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:48:56,625 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:48:56,625 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 01:48:56,625 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:48:56,625 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:48:56,625 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 01:48:56,625 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:48:56,625 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:48:56,625 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=179} 01:48:56,625 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:48:56,625 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,625 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 01:48:56,625 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:48:56,625 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=183} 01:48:56,625 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:48:56,625 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,625 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:48:56,625 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testReadWriteTransactionWithMultipleShards_member-1-shard-cars-testReadWriteTransactionWithMultipleShards%232028196473$g] 01:48:56,625 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:48:56,625 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:48:56,625 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testReadWriteTransactionWithMultipleShards_member-1-shard-people-testReadWriteTransactionWithMultipleShards%23-729046024$h] 01:48:56,625 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#145002523] with name member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher 01:48:56,625 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-973545804] with name member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher 01:48:56,626 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#145002523] 01:48:56,626 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:48:56,626 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:48:56,626 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:48:56,626 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:56,626 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:56,626 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:48:56,626 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:48:56,626 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 116.1 μs 01:48:56,626 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-973545804] 01:48:56,626 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:48:56,626 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#145002523] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:56,626 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 37.91 μs 01:48:56,627 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit succeeded 01:48:56,627 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:48:56,627 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] for shard cars, trying to use DirectTransactionContextWrapper 01:48:56,627 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:48:56,627 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] found - trying create transaction 01:48:56,627 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] 01:48:56,627 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Queuing TransactionOperation 01:48:56,627 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 1 01:48:56,627 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:48:56,627 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 01:48:56,627 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:48:56,627 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=179} 01:48:56,627 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,628 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 1 01:48:56,628 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1429714] with name member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher 01:48:56,628 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:48:56,628 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 01:48:56,628 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:48:56,628 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:48:56,628 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=183} 01:48:56,628 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:48:56,628 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:48:56,628 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1429714] 01:48:56,628 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:48:56,628 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:48:56,628 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#432997589] with name member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher 01:48:56,628 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:48:56,628 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1429714] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:56,628 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:48:56,628 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 79.82 μs 01:48:56,629 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#432997589] 01:48:56,629 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_19#883130766] 01:48:56,629 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:48:56,629 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 71.88 μs 01:48:56,629 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#432997589] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:56,630 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_19#883130766, transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0, version=13] 01:48:56,631 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:48:56,631 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_19#883130766)] 01:48:56,632 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 ReadData operation succeeded 01:48:56,632 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] for shard people, trying to use DirectTransactionContextWrapper 01:48:56,632 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:48:56,632 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] found - trying create transaction 01:48:56,632 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] 01:48:56,632 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Queuing TransactionOperation 01:48:56,632 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:48:56,633 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_20#-923029426] 01:48:56,633 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_20#-923029426, transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0, version=13] 01:48:56,633 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 01:48:56,633 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_20#-923029426)] 01:48:56,634 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 ReadData operation succeeded 01:48:56,634 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:48:56,635 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:56,635 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:48:56,635 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onShutDown 01:48:56,635 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Initiating leader transfer 01:48:56,635 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithMultipleShards - waiting on 2 shards 01:48:56,635 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onShutDown 01:48:56,635 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Initiating leader transfer 01:48:56,635 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@58529e2d 01:48:56,635 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@58529e2d 01:48:56,635 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 01:48:56,635 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:56,635 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:48:56,635 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6f960b12 01:48:56,635 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6f960b12 01:48:56,635 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 01:48:56,635 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:56,635 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:56,635 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:48:56,635 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:56,635 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:56,635 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:56,636 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards] 01:48:56,636 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards] 01:48:56,636 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:48:56,636 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:48:56,636 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:48:56,636 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:48:56,636 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:48:56,636 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:48:56,636 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:48:56,636 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:48:56,636 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testReadWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:48:56,636 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:48:56,636 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:56,636 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:48:56,636 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Leader transfer complete 01:48:56,636 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Leader transfer complete 01:48:57,50 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 01:48:57,50 PM [cluster-test-akka.actor.default-dispatcher-19] [WARN] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#2028196473]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:48:57,51 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:57,51 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:48:57,51 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader)_slicer-id-505: Closing 01:48:57,51 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader)_slicer-id-506: Closing 01:48:57,51 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:48:57,51 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:48:57,51 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 01:48:57,52 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 01:48:57,52 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:48:57,52 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: shutting down 01:48:57,52 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 01:48:57,52 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 01:48:57,53 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:57,53 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:57,64 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:57,64 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:57,64 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:57,64 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:57,73 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 01:48:57,73 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:48:57,73 PM [cluster-test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Closing 01:48:57,73 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:48:57,73 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Closing 01:48:57,74 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:48:57,74 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:48:57,74 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 01:48:57,76 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:57,76 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:58,67 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4356589091775288550] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:58,87 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-671284327394077265] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:48:58,93 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:58,94 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:58,96 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:58.097] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:58,112 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:58,112 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/11/2024 13:48:58.116] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:58,146 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:58,152 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [3677453583708775968] 01:48:58,152 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:48:58,153 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:48:58,153 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:48:58,154 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:58,168 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:58,168 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:48:58,168 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:48:58,169 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:48:58,184 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:48:58,195 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [677146966822735312] 01:48:58,195 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:48:58,198 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:58,198 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:48:58,198 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:58,198 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:58,232 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:58,269 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-8860269443799724587] 01:48:58,269 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:48:58,269 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:58,269 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:48:58,269 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:58,282 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:58,282 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:58,298 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:58,331 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:48:58,333 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:58,333 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:48:58,337 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:48:58,358 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:58,358 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:59,375 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:59.376] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:59,384 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:48:59,384 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:48:59,386 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:48:59.387] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:59,410 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:59,415 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-8261695645570502440] 01:48:59,418 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:48:59,418 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:48:59,418 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:48:59,423 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:59,423 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:59,426 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:48:59,426 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:48:59,426 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:48:59,434 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 closeTransaction called 01:48:59,434 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_19#883130766)] 01:48:59,434 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 closeTransaction called 01:48:59,434 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_20#-923029426)] 01:48:59,434 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 closeTransaction called 01:48:59,434 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_15#-1149360644)] 01:48:59,439 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:48:59,450 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [835071705068208748] 01:48:59,450 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:48:59,453 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:59,453 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:48:59,457 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:59,457 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:59,474 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:48:59,482 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-456026608298685641] 01:48:59,482 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:48:59,501 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:59,501 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:48:59,501 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:48:59,503 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:59,504 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleTransactionsWritesInQuickSuccession 01:48:59,505 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:48:59,505 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:48:59,511 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-832758374] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:59,512 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 01:48:59,515 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testSingleTransactionsWritesInQuickSuccession 01:48:59,515 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Creating local shard: member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:48:59,515 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: no initial generation override, starting from 0 01:48:59,515 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:48:59,515 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@69605f1 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4c7e5a5 01:48:59,515 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:59,515 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:48:59,520 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1fdd9864 01:48:59,520 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, 0, 1710164939515, None)) 01:48:59,521 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164939514,0,0)) 01:48:59,521 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4c7e5a5 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2ad0034a 01:48:59,521 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@dace1b 01:48:59,521 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} started 01:48:59,522 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:59,523 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:48:59,525 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleTransactionsWritesInQuickSuccession 01:48:59,526 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:59,526 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:48:59,526 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:48:59,526 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:48:59,526 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: updated peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#502687832] 01:48:59,527 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, persistent : true 01:48:59,528 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#502687832] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:59,528 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with journal batch size 1 01:48:59,528 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-1:snapshot-read#1043787460] 01:48:59,528 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#1082949962 created and ready for shard:member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:48:59,528 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#1082949962] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:59,529 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:59,529 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:59,529 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Initializing shard [cars] 01:48:59,529 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is initialized 01:48:59,529 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 01:48:59,529 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 01:48:59,529 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from null to Follower 01:48:59,529 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , registered listener akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession 01:48:59,529 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from null to Follower 01:48:59,532 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:48:59,537 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:59,542 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 01:48:59,543 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-1:snapshot-read#1043787460] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:59,552 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#885990976] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:59,556 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:48:59,558 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: no initial generation override, starting from 0 01:48:59,558 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:48:59,558 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@22d3a0cf to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@190b51b0 01:48:59,558 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7fb4a9b0 01:48:59,558 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:48:59,558 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, 0, 1710164939558, None)) 01:48:59,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164939557,0,0)) 01:48:59,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@190b51b0 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@23050569 01:48:59,559 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} started 01:48:59,560 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSingleTransactionsWritesInQuickSuccession 01:48:59,560 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Creating local shard: member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:48:59,560 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:48:59,560 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:48:59,560 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@29c2c534 01:48:59,560 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:48:59,560 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:48:59,561 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:48:59,561 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:48:59,561 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: updated peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession#1794566617] 01:48:59,561 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, persistent : true 01:48:59,561 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession#1794566617] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:59,562 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#-595346052 created and ready for shard:member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:48:59,562 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#-595346052] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:59,562 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-2:snapshot-read#-1217858123] 01:48:59,562 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with journal batch size 1 01:48:59,562 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-2:snapshot-read#-1217858123] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:48:59,563 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:48:59,563 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:48:59,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Initializing shard [cars] 01:48:59,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession is initialized 01:48:59,563 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 01:48:59,563 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 01:48:59,563 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from null to Follower 01:48:59,563 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:48:59,563 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession , registered listener akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession 01:48:59,563 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession from null to Follower 01:48:59,609 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-832758374] 01:48:59,610 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: findLocalShard : cars 01:48:59,610 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#502687832] 01:48:59,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:48:59,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Received ElectionTimeout - switching to Candidate 01:48:59,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] 01:48:59,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:48:59,799 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 1 01:48:59,799 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:48:59,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Closing 01:48:59,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 01:48:59,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 01:48:59,800 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from Follower to Candidate 01:48:59,800 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from Follower to Candidate 01:49:00,89 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 01:49:00,89 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,89 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 2 01:49:00,127 PM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:00,378 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 01:49:00,378 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,378 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 3 01:49:00,440 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:49:00,440 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:00,441 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:00,441 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,441 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,441 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:00,441 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,441 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,441 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,449 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:00,449 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,449 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,449 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:00,449 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,449 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,514 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:49:00,649 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 01:49:00,649 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,649 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 4 01:49:00,649 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,651 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:00,651 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Cluster state: [] 01:49:00,651 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:00,651 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:00,651 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=null 01:49:00,651 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:00,651 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,651 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:49:00,654 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:49:00,654 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:49:00,654 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] 01:49:00,654 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:00,654 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:49:00,654 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: true, hasLeader : true 01:49:00,654 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: became leader with frontend state for [] 01:49:00,654 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from Candidate to Leader 01:49:00,654 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderPayloadVersion=13] 01:49:00,654 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:00,654 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:00,654 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from Candidate to Leader 01:49:00,654 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All Shards are ready - data store testSingleTransactionsWritesInQuickSuccession is ready 01:49:00,655 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Closing 01:49:00,655 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: getLeaderAddress leaderId = member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,655 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderPayloadVersion=13] 01:49:00,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:00,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:00,655 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All Shards are ready - data store testSingleTransactionsWritesInQuickSuccession is ready 01:49:00,655 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession Received follower initial sync status for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession status sync done false 01:49:00,656 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: -1, nextIndex: 0 01:49:00,656 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:00,665 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testSingleTransactionsWritesInQuickSuccession commit rateLimit adjusted to 40.88563184859886 01:49:00,665 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:49:00,665 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#885990976] 01:49:00,665 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:49:00,665 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:49:00,665 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Readying 1 components for commit 01:49:00,665 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 01:49:00,665 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:49:00,665 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 canCommit 01:49:00,666 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:00,666 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, primaryVersion=13] 01:49:00,666 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, primaryVersion=13] 01:49:00,666 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars 01:49:00,666 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 01:49:00,666 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:49:00,666 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:49:00,666 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 directCommit called 01:49:00,666 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 sending 2 batched modifications, ready: true 01:49:00,666 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:49:00,669 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, disabling transaction tracking 01:49:00,669 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 01:49:00,669 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 01:49:00,669 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:49:00,669 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:00,669 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,669 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}]]] 01:49:00,669 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 01:49:00,669 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 01:49:00,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:49:00,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:49:00,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@74dc8 01:49:00,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 2 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:49:00,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 of 2 operations, client version 13 01:49:00,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:49:00,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 ready 01:49:00,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 01:49:00,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:49:00,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 validated 01:49:00,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:00,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: success 01:49:00,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:49:00,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:49:00,671 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 candidate ready 01:49:00,671 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: canCommit - candidate rootPath: / 01:49:00,671 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: canCommit - messages: [] 01:49:00,671 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: preCommit - successfulFromPrevious: [] 01:49:00,671 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 prepared 01:49:00,671 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:00,671 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:49:00,671 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:49:00,671 PM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 183 01:49:00,671 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 01:49:00,671 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 01:49:00,671 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:00,671 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:49:00,671 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,671 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 submitted to persistence 01:49:00,672 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 01:49:00,672 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 01:49:00,672 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 01:49:00,672 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}]]] 01:49:00,672 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:49:00,672 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:49:00,672 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72} 01:49:00,672 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 01:49:00,672 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 1 01:49:00,672 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:00,673 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 01:49:00,673 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession Received follower initial sync status for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession status sync done true 01:49:00,674 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 0, nextIndex: 1 01:49:00,674 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:49:00,674 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 0 01:49:00,674 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from -1 to 0 01:49:00,674 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72} 01:49:00,674 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:49:00,674 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} does not match any client, pre-disabling client. 01:49:00,674 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, nanosAgo=5529320} is already disabled 01:49:00,675 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:49:00,675 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,675 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}]]] 01:49:00,675 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=0 01:49:00,675 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}]]] 01:49:00,675 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 01:49:00,675 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:49:00,675 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74} 01:49:00,675 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 01:49:00,676 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=183} 01:49:00,676 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 01:49:00,676 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 01:49:00,676 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 0 01:49:00,676 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:00,676 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 2 01:49:00,676 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 0 01:49:00,676 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from -1 to 0 01:49:00,676 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72} 01:49:00,676 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:49:00,676 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} does not match any client, pre-disabling client. 01:49:00,676 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 01:49:00,676 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 01:49:00,676 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=0 01:49:00,676 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 2, nextIndex: 3 01:49:00,676 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 01:49:00,676 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 1 01:49:00,676 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 0 to 1 01:49:00,676 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74} 01:49:00,676 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} has no callback 01:49:00,676 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 2 01:49:00,676 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 1 to 2 01:49:00,676 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 2 data Simple{size=183} 01:49:00,676 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:49:00,676 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: commit - successfulFromPrevious: [] 01:49:00,676 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:49:00,676 PM [cluster-test-shard-dispatcher-15] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 01:49:00,676 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$a] 01:49:00,677 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-351095595] with name member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher 01:49:00,677 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 1 for term 4 01:49:00,677 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:49:00,677 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=2 01:49:00,677 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-351095595] 01:49:00,677 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:49:00,677 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 59.65 μs 01:49:00,677 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-351095595] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:00,679 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 modifications completed with CommitTransactionReply [version=13] 01:49:00,679 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 successfully completed direct commit 01:49:00,679 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:49:00,679 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:49:00,679 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:49:00,679 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 01:49:00,679 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}] 01:49:00,679 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 Readying 1 components for commit 01:49:00,679 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 Readying transaction for shard cars 01:49:00,679 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 directCommit called 01:49:00,679 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 sending 1 batched modifications, ready: true 01:49:00,679 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:49:00,679 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 canCommit 01:49:00,680 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 2 01:49:00,680 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 01:49:00,680 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 1 01:49:00,680 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 0 to 1 01:49:00,680 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74} 01:49:00,680 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 2 01:49:00,680 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 1 to 2 01:49:00,680 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 2 data Simple{size=183} 01:49:00,680 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:49:00,680 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-1800025542] with name member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher 01:49:00,680 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 1 for term 4 01:49:00,680 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=2 01:49:00,680 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-1800025542] 01:49:00,680 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:49:00,680 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-1800025542] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:00,680 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 82.45 μs 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@785164fa 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 of 1 operations, client version 13 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 ready 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}], operation=WRITE}}}}}}}]} 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 validated 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: success 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 candidate ready 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: canCommit - candidate rootPath: / 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: canCommit - messages: [] 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: preCommit - successfulFromPrevious: [] 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 prepared 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:49:00,681 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:49:00,682 PM [cluster-test-shard-dispatcher-15] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 01:49:00,682 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 01:49:00,682 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 01:49:00,682 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:00,682 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 01:49:00,682 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,682 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}]]] 01:49:00,682 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 submitted to persistence 01:49:00,682 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 01:49:00,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}]]] 01:49:00,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:49:00,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:49:00,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 01:49:00,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 01:49:00,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 01:49:00,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:00,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 01:49:00,684 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 3, nextIndex: 4 01:49:00,684 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 01:49:00,684 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 3 01:49:00,684 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 2 to 3 01:49:00,684 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 3 data Simple{size=240} 01:49:00,684 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:49:00,684 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: commit - successfulFromPrevious: [] 01:49:00,684 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 01:49:00,684 PM [cluster-test-shard-dispatcher-15] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:49:00,684 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$b] 01:49:00,684 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 2 for term 4 01:49:00,684 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:49:00,684 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=3 01:49:00,684 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:49:00,684 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 38.31 μs 01:49:00,690 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 modifications completed with CommitTransactionReply [version=13] 01:49:00,690 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 successfully completed direct commit 01:49:00,690 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 01:49:00,690 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:49:00,690 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:49:00,690 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 01:49:00,690 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:49:00,690 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 Queuing TransactionOperation 01:49:00,690 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 3 01:49:00,690 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 01:49:00,690 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 3 01:49:00,690 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 2 to 3 01:49:00,690 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 3 data Simple{size=240} 01:49:00,690 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:49:00,691 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 2 for term 4 01:49:00,691 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=3 01:49:00,691 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:49:00,691 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 01:49:00,691 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:49:00,691 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@13d063a9 01:49:00,691 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 116.5 μs 01:49:00,691 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_21#1571702560] 01:49:00,691 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_21#1571702560] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:00,692 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_21#1571702560, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0, version=13] 01:49:00,692 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:49:00,692 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_21#1571702560)] 01:49:00,694 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 ReadData operation succeeded 01:49:00,694 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 closeTransaction called 01:49:00,694 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_21#1571702560)] 01:49:00,694 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:49:00,694 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:49:00,694 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:49:00,694 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 01:49:00,694 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}] 01:49:00,694 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 Readying 1 components for commit 01:49:00,694 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 Readying transaction for shard cars 01:49:00,694 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 directCommit called 01:49:00,694 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 sending 1 batched modifications, ready: true 01:49:00,694 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:49:00,695 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 canCommit 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@13d063a9 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 of 1 operations, client version 13 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 ready 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}], operation=WRITE}}}}}}}]} 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 validated 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: success 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 candidate ready 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: canCommit - candidate rootPath: / 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: canCommit - messages: [] 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: preCommit - successfulFromPrevious: [] 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 prepared 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:49:00,696 PM [cluster-test-shard-dispatcher-15] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 01:49:00,697 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 01:49:00,697 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 01:49:00,697 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:00,697 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 2 01:49:00,697 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,697 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}]]] 01:49:00,697 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 submitted to persistence 01:49:00,697 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 01:49:00,698 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}]]] 01:49:00,698 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:49:00,698 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 01:49:00,698 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 01:49:00,698 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 01:49:00,698 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 01:49:00,698 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:00,698 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 01:49:00,699 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 4, nextIndex: 5 01:49:00,699 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 01:49:00,699 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 4 01:49:00,699 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 3 to 4 01:49:00,699 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 4 data Simple{size=240} 01:49:00,699 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:49:00,699 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: commit - successfulFromPrevious: [] 01:49:00,699 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 changing state from COMMIT_PENDING to COMMITTED 01:49:00,699 PM [cluster-test-shard-dispatcher-15] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:49:00,699 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$e] 01:49:00,699 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 3 for term 4 01:49:00,699 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:49:00,699 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=4 01:49:00,699 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:49:00,700 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 57.36 μs 01:49:00,700 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 successfully completed direct commit 01:49:00,700 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 modifications completed with CommitTransactionReply [version=13] 01:49:00,700 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 01:49:00,700 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:49:00,700 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:49:00,700 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 01:49:00,700 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:49:00,700 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 Queuing TransactionOperation 01:49:00,706 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 01:49:00,706 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:49:00,706 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@4f667011 01:49:00,706 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_22#1468631646] 01:49:00,706 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_22#1468631646] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:00,706 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 4 01:49:00,707 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4 01:49:00,707 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 4 01:49:00,707 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 3 to 4 01:49:00,707 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 4 data Simple{size=240} 01:49:00,707 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:49:00,707 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 3 for term 4 01:49:00,707 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=4 01:49:00,707 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_22#1468631646, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0, version=13] 01:49:00,707 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:49:00,707 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_22#1468631646)] 01:49:00,707 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:49:00,707 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 58.29 μs 01:49:00,713 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 ReadData operation succeeded 01:49:00,714 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 closeTransaction called 01:49:00,714 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_22#1468631646)] 01:49:00,714 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:49:00,714 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:49:00,714 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:49:00,714 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 01:49:00,714 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}] 01:49:00,714 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 Readying 1 components for commit 01:49:00,714 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 Readying transaction for shard cars 01:49:00,714 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 directCommit called 01:49:00,714 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 sending 1 batched modifications, ready: true 01:49:00,714 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:49:00,714 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 canCommit 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@4f667011 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 of 1 operations, client version 13 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 ready 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}], operation=WRITE}}}}}}}]} 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 validated 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: success 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 candidate ready 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: canCommit - candidate rootPath: / 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: canCommit - messages: [] 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: preCommit - successfulFromPrevious: [] 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 prepared 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:49:00,718 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:49:00,719 PM [cluster-test-shard-dispatcher-15] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 01:49:00,719 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 01:49:00,719 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 01:49:00,719 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:00,719 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 3 01:49:00,719 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,719 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}]]] 01:49:00,719 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 submitted to persistence 01:49:00,719 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 01:49:00,720 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}]]] 01:49:00,720 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:49:00,720 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 01:49:00,720 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 01:49:00,720 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 01:49:00,720 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 01:49:00,720 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=5, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:00,720 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 01:49:00,721 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 5, nextIndex: 6 01:49:00,721 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 01:49:00,721 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 5 01:49:00,721 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 4 to 5 01:49:00,721 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 5 data Simple{size=240} 01:49:00,721 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:49:00,721 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: commit - successfulFromPrevious: [] 01:49:00,721 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 changing state from COMMIT_PENDING to COMMITTED 01:49:00,721 PM [cluster-test-shard-dispatcher-15] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:49:00,721 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$h] 01:49:00,721 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 4 for term 4 01:49:00,721 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 01:49:00,721 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=5 01:49:00,722 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:49:00,722 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 54.82 μs 01:49:00,722 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 successfully completed direct commit 01:49:00,722 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 modifications completed with CommitTransactionReply [version=13] 01:49:00,723 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 01:49:00,723 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:49:00,723 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:49:00,723 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 01:49:00,723 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:49:00,723 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 Queuing TransactionOperation 01:49:00,723 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 5 01:49:00,723 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5 01:49:00,723 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 5 01:49:00,723 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 4 to 5 01:49:00,723 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 5 data Simple{size=240} 01:49:00,723 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:49:00,723 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 4 for term 4 01:49:00,723 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=5 01:49:00,724 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:49:00,724 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 46.45 μs 01:49:00,725 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 01:49:00,725 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:49:00,725 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@7d08f6aa 01:49:00,725 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_23#-615711525] 01:49:00,725 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_23#-615711525] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:00,726 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_23#-615711525, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0, version=13] 01:49:00,726 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:49:00,726 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_23#-615711525)] 01:49:00,728 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 ReadData operation succeeded 01:49:00,728 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 closeTransaction called 01:49:00,728 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_23#-615711525)] 01:49:00,728 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:49:00,728 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:49:00,728 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:49:00,728 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 01:49:00,728 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}] 01:49:00,728 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 Readying 1 components for commit 01:49:00,728 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 Readying transaction for shard cars 01:49:00,728 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 directCommit called 01:49:00,728 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 sending 1 batched modifications, ready: true 01:49:00,728 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:49:00,728 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 canCommit 01:49:00,729 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:49:00,729 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@7d08f6aa 01:49:00,729 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:49:00,729 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 of 1 operations, client version 13 01:49:00,729 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:49:00,729 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 ready 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}], operation=WRITE}}}}}}}]} 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 validated 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: success 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 candidate ready 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: canCommit - candidate rootPath: / 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: canCommit - messages: [] 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: preCommit - successfulFromPrevious: [] 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 prepared 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 6, leaderLastIndex: 6, leaderSnapShotIndex: 4 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 6 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}]]] 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 submitted to persistence 01:49:00,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 01:49:00,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}]]] 01:49:00,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:49:00,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 5, entries to be added from: 0 01:49:00,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 01:49:00,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 01:49:00,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 01:49:00,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:00,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 01:49:00,733 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 6, nextIndex: 7 01:49:00,733 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 5 01:49:00,733 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 6 01:49:00,733 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 5 to 6 01:49:00,733 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 6 data Simple{size=240} 01:49:00,733 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:49:00,733 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: commit - successfulFromPrevious: [] 01:49:00,733 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 changing state from COMMIT_PENDING to COMMITTED 01:49:00,733 PM [cluster-test-shard-dispatcher-15] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:49:00,733 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$k] 01:49:00,733 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 5 for term 4 01:49:00,733 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 01:49:00,733 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=6 01:49:00,733 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:49:00,733 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 64.38 μs 01:49:00,734 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 successfully completed direct commit 01:49:00,734 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 modifications completed with CommitTransactionReply [version=13] 01:49:00,734 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 01:49:00,734 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:49:00,734 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:49:00,734 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 01:49:00,734 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:49:00,734 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 Queuing TransactionOperation 01:49:00,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 6 01:49:00,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 5, lastIndex(): 6 01:49:00,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 6 01:49:00,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 5 to 6 01:49:00,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 6 data Simple{size=240} 01:49:00,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:49:00,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 5 for term 4 01:49:00,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=6 01:49:00,735 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:49:00,735 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 69.16 μs 01:49:00,737 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 01:49:00,737 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:49:00,737 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@39443315 01:49:00,737 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_24#990179861] 01:49:00,737 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_24#990179861] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:00,738 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_24#990179861, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0, version=13] 01:49:00,738 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:49:00,738 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_24#990179861)] 01:49:00,739 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 ReadData operation succeeded 01:49:00,740 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 closeTransaction called 01:49:00,740 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_24#990179861)] 01:49:00,740 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:49:00,740 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:49:00,740 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:49:00,740 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 01:49:00,740 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}] 01:49:00,740 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 Readying 1 components for commit 01:49:00,740 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 Readying transaction for shard cars 01:49:00,740 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 directCommit called 01:49:00,740 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 sending 1 batched modifications, ready: true 01:49:00,740 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:49:00,740 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 canCommit 01:49:00,741 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:49:00,741 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@39443315 01:49:00,741 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:49:00,741 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 of 1 operations, client version 13 01:49:00,741 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:49:00,741 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 ready 01:49:00,741 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}], operation=WRITE}}}}}}}]} 01:49:00,741 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:49:00,741 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 validated 01:49:00,741 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:00,741 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: success 01:49:00,741 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:49:00,741 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:49:00,742 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 candidate ready 01:49:00,742 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: canCommit - candidate rootPath: / 01:49:00,742 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: canCommit - messages: [] 01:49:00,742 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: preCommit - successfulFromPrevious: [] 01:49:00,742 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 prepared 01:49:00,742 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:00,742 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:49:00,742 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:49:00,742 PM [cluster-test-shard-dispatcher-15] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 01:49:00,742 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 01:49:00,742 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 01:49:00,742 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:00,742 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: 5 01:49:00,742 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 7 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:00,742 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}]]] 01:49:00,742 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 submitted to persistence 01:49:00,743 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 01:49:00,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}]]] 01:49:00,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:49:00,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 01:49:00,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 01:49:00,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 01:49:00,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 01:49:00,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=7, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:00,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 01:49:00,744 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 7, nextIndex: 8 01:49:00,744 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 6 01:49:00,744 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 7 01:49:00,744 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 6 to 7 01:49:00,744 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 7 data Simple{size=240} 01:49:00,744 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:49:00,744 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: commit - successfulFromPrevious: [] 01:49:00,744 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 changing state from COMMIT_PENDING to COMMITTED 01:49:00,744 PM [cluster-test-shard-dispatcher-15] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:49:00,744 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$n] 01:49:00,744 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 6 for term 4 01:49:00,744 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 01:49:00,744 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=7 01:49:00,745 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:49:00,745 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 56.31 μs 01:49:00,746 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 successfully completed direct commit 01:49:00,746 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 modifications completed with CommitTransactionReply [version=13] 01:49:00,746 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 01:49:00,746 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:49:00,746 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:49:00,746 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 01:49:00,746 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:49:00,746 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 7 01:49:00,746 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 7, context.getLastApplied(): 6, lastIndex(): 7 01:49:00,746 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 7 01:49:00,746 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 6 to 7 01:49:00,746 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 7 data Simple{size=240} 01:49:00,746 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:49:00,747 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 6 for term 4 01:49:00,747 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=7 01:49:00,747 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:49:00,747 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 70.62 μs 01:49:00,748 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 Queuing TransactionOperation 01:49:00,750 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 01:49:00,750 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:49:00,750 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@717862f0 01:49:00,750 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_25#513089437] 01:49:00,750 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_25#513089437] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:00,751 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_25#513089437, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0, version=13] 01:49:00,751 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:49:00,751 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_25#513089437)] 01:49:00,754 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 ReadData operation succeeded 01:49:00,754 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 closeTransaction called 01:49:00,754 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_25#513089437)] 01:49:01,256 PM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-832758374] 01:49:01,256 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: findLocalShard : cars 01:49:01,256 PM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#502687832] 01:49:01,256 PM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.TestShard$RequestFrontendMetadata to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#502687832] 01:49:01,257 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 01:49:01,257 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:49:01,257 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:49:01,257 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 01:49:01,257 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:49:01,258 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 Queuing TransactionOperation 01:49:01,259 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 01:49:01,259 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:49:01,260 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@717862f0 01:49:01,260 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-11_26#-171196661] 01:49:01,261 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-11_26#-171196661, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0, version=13] 01:49:01,261 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:49:01,261 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-11_26#-171196661)] 01:49:01,265 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 ReadData operation succeeded 01:49:01,265 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 closeTransaction called 01:49:01,265 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-11_26#-171196661)] 01:49:01,266 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:49:01,266 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Issuing gracefulStop to shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:01,266 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession - waiting on 1 shards 01:49:01,266 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onShutDown 01:49:01,266 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Initiating leader transfer 01:49:01,266 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@79062800 01:49:01,266 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@79062800 01:49:01,266 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Attempting to transfer leadership 01:49:01,266 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 01:49:01,266 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=null, leaderPayloadVersion=13] 01:49:01,266 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:01,266 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:01,275 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession] 01:49:01,275 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=null, leaderPayloadVersion=13] 01:49:01,275 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:01,275 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:01,276 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, matchIndex: 7, lastIndex: 7, isVoting: true 01:49:01,276 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Follower's log matches - sending ElectionTimeout 01:49:01,276 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 01:49:01,276 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Leader transfer complete 01:49:01,460 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:49:01,491 PM [cluster-test-shard-dispatcher-12] [WARN] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=7, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 212, lastApplied : 7, commitIndex : 7 01:49:01,678 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All shards shutdown - sending PoisonPill to self 01:49:01,678 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#502687832]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:49:01,679 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:01,679 PM [cluster-test-shard-dispatcher-15] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader)_slicer-id-511: Closing 01:49:01,679 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborting 0 pending queued transactions 01:49:01,679 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 01:49:01,679 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 01:49:01,679 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:49:01,679 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: shutting down 01:49:01,679 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 01:49:01,680 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 01:49:01,680 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:01,680 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:01,691 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:01,691 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:01,691 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:01,692 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:01,694 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:01,714 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 01:49:01,714 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:49:01,714 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Closing 01:49:01,715 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborting 0 pending queued transactions 01:49:01,715 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 01:49:01,715 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:01,715 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:02,704 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-8261695645570502440] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:02,730 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:02,730 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:02,734 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [835071705068208748] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:02,737 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:02.738] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:02,750 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:02,750 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/11/2024 13:49:02.753] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:02,775 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:02,779 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-7807297777996250746] 01:49:02,780 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:02,782 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:02,782 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:02,790 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:02,790 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:02,795 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:02,795 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:02,795 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:02,801 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:02,810 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4590229043598439985] 01:49:02,810 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:02,812 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:02,812 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:02,819 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:02,819 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:02,828 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:02,832 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-56862052605311200] 01:49:02,832 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:02,834 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:02,834 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:02,842 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:02,843 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:02,846 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 01:49:02,848 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:02,848 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:02,855 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:49:02,856 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1223746217] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:02,859 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 01:49:02,859 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:02,859 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:02,859 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:02,860 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 01:49:02,860 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:49:02,860 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@11c65677 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4991c831 01:49:02,860 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@c934f67 01:49:02,863 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1710164942862, None)) 01:49:02,863 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164942861,0,0)) 01:49:02,864 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@452c57a0 01:49:02,864 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4991c831 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@67bede05 01:49:02,864 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 01:49:02,864 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:02,866 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:02,867 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 01:49:02,868 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:02,868 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:02,868 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:02,868 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:02,868 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1909847204] 01:49:02,868 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 01:49:02,869 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1909847204] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:02,869 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#383799725 created and ready for shard:member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:02,869 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-1:snapshot-read#-866309118] 01:49:02,869 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#383799725] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:02,869 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 01:49:02,869 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-1:snapshot-read#-866309118] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:02,871 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:02,871 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:02,871 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 01:49:02,871 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 01:49:02,871 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:49:02,871 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:49:02,872 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 01:49:02,872 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 01:49:02,872 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 01:49:02,887 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-128910126] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:02,888 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:49:02,890 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 01:49:02,890 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:49:02,890 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@291359f7 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@187877c8 01:49:02,890 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@bf0e7dd 01:49:02,890 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1710164942890, None)) 01:49:02,891 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164942889,0,0)) 01:49:02,891 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@187877c8 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@37cd929 01:49:02,891 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 01:49:02,892 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 01:49:02,892 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:02,892 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@564ec4d5 01:49:02,892 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:02,892 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:02,893 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 01:49:02,893 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop#376190883] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:02,893 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:49:02,893 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#326921051 created and ready for shard:member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:02,894 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-2:snapshot-read#1373080101] 01:49:02,894 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#326921051] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:02,894 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-2:snapshot-read#1373080101] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:02,895 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 01:49:02,896 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:02,896 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:02,896 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:49:02,896 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:49:02,896 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 01:49:02,896 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 01:49:02,897 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 01:49:02,897 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 01:49:02,897 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 01:49:02,898 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:02,899 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:02,903 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:02,903 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:02,904 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:02,904 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:02,904 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:02,942 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1223746217] 01:49:02,942 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: findLocalShard : cars 01:49:02,942 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1909847204] 01:49:03,182 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:03,182 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Received ElectionTimeout - switching to Candidate 01:49:03,182 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:49:03,182 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=1, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:03,182 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 1 01:49:03,182 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:03,182 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 01:49:03,182 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:49:03,182 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:49:03,182 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Follower to Candidate 01:49:03,182 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Follower to Candidate 01:49:03,435 PM [cluster-test-shard-dispatcher-23] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:03,473 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 01:49:03,473 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=2, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:03,473 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 2 01:49:03,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 01:49:03,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=3, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:03,723 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 3 01:49:03,804 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:49:03,804 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:03,804 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:03,804 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:03,804 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:03,804 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:03,804 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:03,804 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:03,804 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:03,804 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:03,810 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:03,810 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:03,810 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:03,810 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:03,810 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:03,810 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:03,810 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:03,857 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:49:03,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 01:49:03,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:03,972 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 4 01:49:03,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:03,973 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:03,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:03,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 01:49:03,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:03,974 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:03,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=null 01:49:03,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:03,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:03,974 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1909847204] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:03,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:49:03,976 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:49:03,976 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:49:03,976 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:49:03,976 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:03,976 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:03,976 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:49:03,976 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 01:49:03,976 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: became leader with frontend state for [] 01:49:03,976 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 01:49:03,976 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:03,976 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:03,976 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Candidate to Leader 01:49:03,976 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Candidate to Leader 01:49:03,976 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 01:49:03,977 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 01:49:03,977 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: getLeaderAddress leaderId = member-1-shard-cars-testRaftCallbackDuringLeadershipDrop peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:03,977 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1909847204] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:03,977 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done false 01:49:03,977 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 01:49:03,977 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:03,977 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:03,977 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 01:49:03,978 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 01:49:03,978 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:03,999 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testRaftCallbackDuringLeadershipDrop commit rateLimit adjusted to 40.88563184859886 01:49:03,999 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:49:03,999 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1223746217] 01:49:03,999 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 Queuing TransactionOperation 01:49:03,999 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 Readying 1 components for commit 01:49:03,999 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 Readying transaction for shard cars 01:49:03,999 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 Queuing TransactionOperation 01:49:03,999 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 canCommit 01:49:03,999 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:03,999 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1909847204, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@643ea2d4] 01:49:03,999 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1909847204, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@643ea2d4] 01:49:03,999 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1909847204)] for shard cars 01:49:03,999 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 01:49:03,999 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@5fee0d4e 01:49:04,0 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 = [ 1 ] 01:49:04,0 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1909847204)] 01:49:04,0 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: handleReadyLocalTransaction for member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:49:04,0 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, disabling transaction tracking 01:49:04,0 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 01:49:04,0 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 01:49:04,0 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}]]] 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying local modifications for Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Validating transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 validated 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: canCommit for member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: success 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Committing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Preparing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 candidate ready 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: canCommit - messages: [] 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 prepared 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Finishing commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Starting commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 173 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:49:04,1 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:04,2 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:04,2 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:04,2 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,2 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 submitted to persistence 01:49:04,2 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}]]] 01:49:04,2 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 1 01:49:04,2 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:49:04,2 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 01:49:04,2 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 01:49:04,2 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 1 01:49:04,2 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:04,2 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1909847204] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:04,2 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 01:49:04,2 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1909847204] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:04,2 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:49:04,3 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 01:49:04,3 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done true 01:49:04,4 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 0, nextIndex: 1 01:49:04,4 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:49:04,4 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Setting last applied to 0 01:49:04,4 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 01:49:04,4 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 01:49:04,4 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:49:04,4 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} does not match any client, pre-disabling client. 01:49:04,4 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, nanosAgo=4091276} is already disabled 01:49:04,4 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:04,4 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,4 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 01:49:04,4 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=0 01:49:04,5 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 01:49:04,5 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 1 01:49:04,5 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:49:04,5 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=173} 01:49:04,5 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:49:04,6 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 2 01:49:04,6 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 0 01:49:04,6 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:04,6 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:49:04,6 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 0 01:49:04,6 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 01:49:04,6 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 01:49:04,6 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:49:04,6 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} does not match any client, pre-disabling client. 01:49:04,9 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:49:04,9 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=0 01:49:04,13 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 1, nextIndex: 2 01:49:04,13 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:49:04,13 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Setting last applied to 1 01:49:04,13 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 01:49:04,13 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data Simple{size=173} 01:49:04,13 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Resuming commit of transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:49:04,13 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:49:04,13 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:49:04,13 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testRaftCallbackDuringLeadershipDrop_member-1-shard-cars-testRaftCallbackDuringLeadershipDrop%231909847204$a] 01:49:04,13 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#299551909] with name member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 01:49:04,14 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:04,14 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 01:49:04,14 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 successfully completed direct commit 01:49:04,14 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 committed successfully 01:49:04,14 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#299551909] 01:49:04,14 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:49:04,14 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#299551909] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:04,14 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 68.21 μs 01:49:04,14 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:04,16 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:04,18 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 01:49:04,20 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 1 01:49:04,20 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:49:04,20 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 1 01:49:04,20 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 01:49:04,20 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data Simple{size=173} 01:49:04,20 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:49:04,21 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-1922554837] with name member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 01:49:04,21 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 01:49:04,21 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1585823552] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:04,21 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-1922554837] 01:49:04,21 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:49:04,22 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 121.7 μs 01:49:04,22 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-1922554837] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:04,23 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:49:04,24 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 01:49:04,24 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:49:04,24 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3ac7c788 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@403ec4f3 01:49:04,25 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 01:49:04,25 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@53c5290 01:49:04,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,25 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1710164944024, None)) 01:49:04,25 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:04,25 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,25 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:04,25 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,25 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:04,25 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,27 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164944023,0,0)) 01:49:04,27 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@403ec4f3 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@15d43099 01:49:04,27 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 01:49:04,28 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1585823552] 01:49:04,33 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3272bf09 01:49:04,33 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:04,33 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,33 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,33 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,33 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#884808339] 01:49:04,33 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,33 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,33 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#884808339] 01:49:04,33 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,33 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,33 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#884808339] 01:49:04,34 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#884808339] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:04,34 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 01:49:04,35 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-1038549827 created and ready for shard:member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,35 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-1038549827] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:04,35 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-3:snapshot-read#800979238] 01:49:04,35 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-3:snapshot-read#800979238] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:04,35 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 01:49:04,35 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-128910126] 01:49:04,36 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:49:04,36 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1909847204)] for shard cars, trying to use DirectTransactionContextWrapper 01:49:04,36 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@5fee0d4e 01:49:04,36 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 01:49:04,38 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1223746217] 01:49:04,38 PM [pool-34-thread-1] [DEBUG] TransactionProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 Readying 1 components for commit 01:49:04,38 PM [pool-34-thread-1] [DEBUG] TransactionProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 Readying transaction for shard cars 01:49:04,38 PM [pool-34-thread-1] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 = [ 1 ] 01:49:04,38 PM [pool-34-thread-1] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1909847204)] 01:49:04,38 PM [pool-34-thread-1] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 canCommit 01:49:04,38 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: handleReadyLocalTransaction for member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying local modifications for Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Validating transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 validated 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: canCommit for member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: success 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Committing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Preparing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 candidate ready 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: canCommit - candidate rootPath: / 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: canCommit - messages: [] 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 prepared 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Finishing commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Starting commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 71 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:49:04,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,40 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}]]] 01:49:04,40 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 submitted to persistence 01:49:04,40 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:04,40 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:04,40 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:49:04,40 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:49:04,40 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 01:49:04,40 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,40 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,40 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 01:49:04,40 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 01:49:04,40 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 01:49:04,40 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 01:49:04,41 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 01:49:04,140 PM [awaitility-thread] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1909847204] 01:49:04,141 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1909847204] 01:49:04,146 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:04,146 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Cluster state: [] 01:49:04,146 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:04,146 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Term 5 in "RequestVote [term=5, candidateId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1]" message is greater than leader's term 4 - switching to Follower 01:49:04,146 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=5, votedFor=null 01:49:04,146 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader) :- Switching from behavior Leader to Follower, election term: 5 01:49:04,146 PM [cluster-test-shard-dispatcher-14] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader)_slicer-id-516: Closing 01:49:04,146 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:49:04,146 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:49:04,146 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: removing frontend state for [member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop] 01:49:04,146 PM [cluster-test-shard-dispatcher-14] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 01:49:04,146 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Leader to Follower 01:49:04,147 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=-1] 01:49:04,147 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:04,147 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:04,147 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Leader to Follower 01:49:04,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:04,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Received ElectionTimeout - switching to Candidate 01:49:04,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:49:04,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,383 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 6 01:49:04,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] to peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] to peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,383 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower) :- Switching from behavior Follower to Candidate, election term: 6 01:49:04,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 01:49:04,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:49:04,384 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:49:04,384 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Follower to Candidate 01:49:04,384 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Follower to Candidate 01:49:04,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:04,385 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:04,385 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 01:49:04,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 01:49:04,385 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 01:49:04,385 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 6 in "RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 4 - updating term 01:49:04,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 01:49:04,385 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=null 01:49:04,385 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 6 in "RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 0 - updating term 01:49:04,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=null 01:49:04,385 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 1, lastTerm: 4 01:49:04,385 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:04,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,385 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 01:49:04,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 01:49:04,387 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=true], my term is 6 01:49:04,387 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=true], current voteCount: 1 01:49:04,387 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): LastApplied index 1 is behind last index 2 - switching to PreLeader 01:49:04,387 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:49:04,387 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:49:04,387 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:49:04,387 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 6 01:49:04,387 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 01:49:04,387 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: became leader with frontend state for [member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop] 01:49:04,387 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:49:04,387 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:49:04,387 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Replicate message: identifier: null, logIndex: 3, payload: class org.opendaylight.controller.cluster.raft.persisted.NoopPayload, isSendImmediate: true 01:49:04,388 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:49:04,388 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:49:04,388 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Candidate to PreLeader 01:49:04,388 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 01:49:04,388 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:04,388 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:04,388 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Candidate to PreLeader 01:49:04,389 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): The followers log is empty and the senders prevLogIndex is 0 01:49:04,389 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:04,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:49:04,389 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 01:49:04,389 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: getLeaderAddress leaderId = member-1-shard-cars-testRaftCallbackDuringLeadershipDrop peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,389 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 01:49:04,389 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:04,389 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:04,389 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 01:49:04,389 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done false 01:49:04,390 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 1, nextIndex: 2 01:49:04,390 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:49:04,390 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,390 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c]]] 01:49:04,391 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,391 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 01:49:04,391 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 01:49:04,391 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 01:49:04,391 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:49:04,391 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,391 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c]]] 01:49:04,397 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): The followers log is empty and the senders prevLogIndex is 0 01:49:04,397 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:04,397 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done true 01:49:04,397 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c]]] 01:49:04,397 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 2 01:49:04,397 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:49:04,397 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=71} 01:49:04,397 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 01:49:04,398 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c 01:49:04,398 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:49:04,398 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 01:49:04,398 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:04,398 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 01:49:04,398 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:49:04,399 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 01:49:04,399 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 01:49:04,399 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,399 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 3, nextIndex: 4 01:49:04,399 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 4 does not match the current term 6 01:49:04,399 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 01:49:04,399 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Setting last applied to 2 01:49:04,399 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 01:49:04,399 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0, size=71} 01:49:04,399 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Resuming commit of transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:49:04,399 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: commit - successfulFromPrevious: [] 01:49:04,399 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 01:49:04,399 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testRaftCallbackDuringLeadershipDrop_member-1-shard-cars-testRaftCallbackDuringLeadershipDrop%231909847204$b] 01:49:04,399 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Setting last applied to 3 01:49:04,399 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 01:49:04,399 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c 01:49:04,399 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:49:04,399 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 successfully completed direct commit 01:49:04,399 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Received ApplyState [identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}]] - lastApplied: 3, lastIndex: 3 01:49:04,400 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 committed successfully 01:49:04,400 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:49:04,400 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:49:04,400 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:49:04,400 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,400 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=3, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c]]] 01:49:04,400 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:49:04,400 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 6 01:49:04,400 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 31.80 μs 01:49:04,400 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader)_slicer-id-519: Closing 01:49:04,400 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 01:49:04,400 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 01:49:04,400 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from PreLeader to Leader 01:49:04,400 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from PreLeader to Leader 01:49:04,400 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 01:49:04,400 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 3 01:49:04,400 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 01:49:04,400 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 2 01:49:04,401 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 01:49:04,401 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{size=71} 01:49:04,401 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:49:04,401 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 3 01:49:04,401 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 01:49:04,401 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c 01:49:04,401 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 01:49:04,401 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:49:04,401 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 24.58 μs 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c]]] 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 4 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=173} 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=71} 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 1 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 3 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 0 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} does not match any client, pre-disabling client. 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 1 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 01:49:04,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data Simple{size=173} 01:49:04,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:49:04,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#1410825067] with name member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 01:49:04,404 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#1410825067] 01:49:04,404 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:49:04,404 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#1410825067] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:04,405 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 91.17 μs 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=3, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c]]] 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 4 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] is present: existingEntryTerm: 4 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] is present: existingEntryTerm: 4 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] is present: existingEntryTerm: 4 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] is present: existingEntryTerm: 6 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 3, entries to be added from: 4 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 3 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 2 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 01:49:04,405 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 3, nextIndex: 4 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{size=71} 01:49:04,405 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 2 for term 6 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:49:04,405 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 3 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 01:49:04,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c 01:49:04,406 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 01:49:04,406 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 01:49:04,406 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:49:04,406 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 18.01 μs 01:49:04,449 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:49:04,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,450 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 01:49:04,450 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 01:49:04,450 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,450 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 01:49:04,450 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 01:49:04,451 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 01:49:04,451 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:49:04,451 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:49:04,452 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:49:04,452 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 01:49:04,452 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,452 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:49:04,453 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 01:49:04,453 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 01:49:04,453 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Initiating leader transfer 01:49:04,453 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2de60eb5 01:49:04,453 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2de60eb5 01:49:04,453 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Attempting to transfer leadership 01:49:04,453 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:49:04,453 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1585823552] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1585823552] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:04,453 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:49:04,453 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=13] 01:49:04,453 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:04,453 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:04,455 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 2 for term 6 01:49:04,455 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:49:04,455 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=13] 01:49:04,455 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:04,455 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:04,456 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, matchIndex: 3, lastIndex: 3, isVoting: true 01:49:04,456 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Follower's log matches - sending ElectionTimeout 01:49:04,456 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:49:04,456 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:49:04,456 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Leader transfer complete 01:49:04,823 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:49:04,873 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 01:49:04,873 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1909847204]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:49:04,873 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,873 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader)_slicer-id-520: Closing 01:49:04,873 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 01:49:04,873 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:49:04,873 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.SendHeartBeat] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1909847204] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1909847204] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1909847204] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:04,873 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:49:04,873 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:49:04,874 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:49:04,874 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 01:49:04,874 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:49:04,875 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:04,875 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/11/2024 13:49:04.880] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-806)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:49:04.880] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-806)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:49:04.880] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-806)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:49:04.881] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-806)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:04,899 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:49:04,900 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:49:04,900 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 01:49:04,900 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 01:49:04,900 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:49:04,901 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:04,901 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:05,896 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-7807297777996250746] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:05,916 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4590229043598439985] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:05,917 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:05,917 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:05,918 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:05.920] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:05,936 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:05,936 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:05,939 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:05.941] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:05,964 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:05,970 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [340122778932299608] 01:49:05,970 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:05,972 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:05,972 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:05,973 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:05,973 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:05,984 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:05,984 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:05,984 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:05,985 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:05,993 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-9049490931874339599] 01:49:05,993 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:05,995 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:05,995 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:05,998 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:05,998 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:06,20 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:06,45 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [651178194151733993] 01:49:06,45 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:06,59 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:06,60 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:06,62 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithSingleShard 01:49:06,64 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:06,64 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:06,64 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:06,64 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:06,68 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#468849845] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:06,68 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithSingleShard 01:49:06,69 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithSingleShard 01:49:06,69 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Creating local shard: member-1-shard-cars-testTransactionChainWithSingleShard 01:49:06,70 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:06,70 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:49:06,76 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:06,79 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:06,83 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: no initial generation override, starting from 0 01:49:06,83 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:49:06,83 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@62c48358 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@49b60a0d 01:49:06,84 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4d8987e3 01:49:06,84 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionChainWithSingleShard, 0, 1710164946084, None)) 01:49:06,84 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164946083,0,0)) 01:49:06,84 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@49b60a0d to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1ed4153b 01:49:06,84 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@87a8121 01:49:06,84 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} started 01:49:06,85 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:06,87 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:06,89 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithSingleShard 01:49:06,90 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:06,90 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionChainWithSingleShard 01:49:06,90 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:49:06,90 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:49:06,90 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: updated peer member-1-shard-cars-testTransactionChainWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard on shard member-1-shard-cars-testTransactionChainWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#858296809] 01:49:06,90 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionChainWithSingleShard, persistent : true 01:49:06,90 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#858296809] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:06,91 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-notifier#-1554543588 created and ready for shard:member-1-shard-cars-testTransactionChainWithSingleShard 01:49:06,91 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-notifier#-1554543588] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:06,91 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-member-1:snapshot-read#-1178866742] 01:49:06,91 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-member-1:snapshot-read#-1178866742] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:06,102 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionChainWithSingleShard with journal batch size 1 01:49:06,103 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:06,103 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:06,103 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:49:06,103 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithSingleShard is no longer the leader 01:49:06,104 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from null to Follower 01:49:06,104 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Initializing shard [cars] 01:49:06,104 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionChainWithSingleShard is initialized 01:49:06,104 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard 01:49:06,104 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from null to Follower 01:49:06,109 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-2054476934] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:06,109 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithSingleShard 01:49:06,113 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: no initial generation override, starting from 0 01:49:06,113 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:49:06,113 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@c364d4a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@14188ee8 01:49:06,125 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithSingleShard 01:49:06,125 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Creating local shard: member-2-shard-cars-testTransactionChainWithSingleShard 01:49:06,125 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5deed251 01:49:06,125 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:06,125 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:49:06,125 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionChainWithSingleShard, 0, 1710164946113, None)) 01:49:06,126 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164946112,0,0)) 01:49:06,126 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@14188ee8 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@22db26ca 01:49:06,126 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6b9308b9 01:49:06,126 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} started 01:49:06,126 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:06,126 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionChainWithSingleShard 01:49:06,126 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:49:06,127 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:49:06,127 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: updated peer member-1-shard-cars-testTransactionChainWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard on shard member-2-shard-cars-testTransactionChainWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard#-533214724] 01:49:06,127 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionChainWithSingleShard, persistent : true 01:49:06,127 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionChainWithSingleShard with journal batch size 1 01:49:06,128 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard#-533214724] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:06,128 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/shard-member-2:snapshot-read#-1430740374] 01:49:06,129 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-notifier#-647939157] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:06,129 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-notifier#-647939157 created and ready for shard:member-2-shard-cars-testTransactionChainWithSingleShard 01:49:06,129 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:49:06,130 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/shard-member-2:snapshot-read#-1430740374] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:06,131 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:06,131 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:06,131 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:49:06,131 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionChainWithSingleShard is no longer the leader 01:49:06,131 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Initializing shard [cars] 01:49:06,131 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionChainWithSingleShard is initialized 01:49:06,131 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithSingleShard , received role change from null to Follower 01:49:06,131 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionChainWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:49:06,131 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard 01:49:06,132 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-2-shard-cars-testTransactionChainWithSingleShard from null to Follower 01:49:06,132 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:06,176 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#468849845] 01:49:06,177 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: findLocalShard : cars 01:49:06,177 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#858296809] 01:49:06,401 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:06,401 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 01:49:06,401 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionChainWithSingleShard] 01:49:06,401 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 01:49:06,402 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 1 01:49:06,402 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:06,402 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Closing 01:49:06,402 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:49:06,402 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithSingleShard is no longer the leader 01:49:06,402 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from Follower to Candidate 01:49:06,402 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from Follower to Candidate 01:49:06,712 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 01:49:06,712 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 01:49:06,712 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 2 01:49:06,713 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:06,993 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:49:06,993 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:06,994 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:06,995 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard 01:49:06,995 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:49:06,995 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:06,995 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:49:06,995 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:49:06,995 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionChainWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:49:07,0 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:07,0 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard 01:49:07,0 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:49:07,0 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:07,0 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:49:07,0 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:49:07,12 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 01:49:07,12 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 01:49:07,12 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 3 01:49:07,12 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionChainWithSingleShard 01:49:07,18 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:07,18 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Cluster state: [] 01:49:07,18 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:07,18 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:07,18 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=3, votedFor=null 01:49:07,18 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:07,18 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 01:49:07,18 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 01:49:07,29 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 01:49:07,29 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 01:49:07,29 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionChainWithSingleShard] 01:49:07,29 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:07,29 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 01:49:07,29 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:49:07,29 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: became leader with frontend state for [] 01:49:07,30 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from Candidate to Leader 01:49:07,30 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderPayloadVersion=13] 01:49:07,30 PM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:07,30 PM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:07,30 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from Candidate to Leader 01:49:07,30 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: All Shards are ready - data store testTransactionChainWithSingleShard is ready 01:49:07,30 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithSingleShard: Closing 01:49:07,30 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testTransactionChainWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:49:07,30 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard Received follower initial sync status for member-2-shard-cars-testTransactionChainWithSingleShard status sync done false 01:49:07,30 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderPayloadVersion=13] 01:49:07,30 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:07,30 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:07,30 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: All Shards are ready - data store testTransactionChainWithSingleShard is ready 01:49:07,31 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionChainWithSingleShard commit rateLimit adjusted to 40.88563184859886 01:49:07,31 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:49:07,31 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-2054476934] 01:49:07,31 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:49:07,31 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Readying 1 components for commit 01:49:07,31 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 01:49:07,31 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:49:07,32 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 01:49:07,32 PM [main] [DEBUG] TransactionChainProxy - Tx: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 - waiting for ready futures with pending Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:49:07,32 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 Queuing TransactionOperation 01:49:07,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:07,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard, primaryVersion=13] 01:49:07,32 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard, primaryVersion=13] 01:49:07,32 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] for shard cars 01:49:07,32 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] found - creating WRITE_ONLY transaction context 01:49:07,32 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:49:07,32 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 directCommit called 01:49:07,32 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 sending 1 batched modifications, ready: true 01:49:07,32 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] 01:49:07,32 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: -1, nextIndex: 0 01:49:07,32 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:07,33 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, disabling transaction tracking 01:49:07,33 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 01:49:07,33 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 01:49:07,33 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:49:07,33 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:07,33 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 01:49:07,33 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}]]] 01:49:07,33 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 01:49:07,33 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 01:49:07,33 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:49:07,33 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} 01:49:07,33 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@2c2e8f73 01:49:07,33 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying 1 batched modifications for Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Readying Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 of 1 operations, client version 13 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 ready 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE}}}]} 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Validating transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 validated 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: canCommit for member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: success 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Committing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Preparing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 candidate ready 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: canCommit - candidate rootPath: / 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: canCommit - messages: [] 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: preCommit - successfulFromPrevious: [] 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 prepared 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Finishing commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Starting commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 173 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=173}] 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=173}] 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 01:49:07,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 submitted to persistence 01:49:07,35 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 01:49:07,35 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 01:49:07,35 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=173}] 01:49:07,35 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}]]] 01:49:07,35 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 1 01:49:07,35 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:49:07,35 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62} 01:49:07,35 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 01:49:07,35 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 1 01:49:07,35 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:07,36 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 01:49:07,36 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard Received follower initial sync status for member-2-shard-cars-testTransactionChainWithSingleShard status sync done true 01:49:07,36 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 0, nextIndex: 1 01:49:07,36 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:49:07,36 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 0 01:49:07,36 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from -1 to 0 01:49:07,36 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62} 01:49:07,36 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:49:07,36 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionChainWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} does not match any client, pre-disabling client. 01:49:07,36 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, nanosAgo=2931848} is already disabled 01:49:07,36 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:49:07,36 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 01:49:07,36 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}], SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=173}]]] 01:49:07,36 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=0 01:49:07,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}], SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=173}]]] 01:49:07,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 2 01:49:07,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:49:07,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64} 01:49:07,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 01:49:07,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log Simple{size=173} 01:49:07,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=173}] 01:49:07,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 3 01:49:07,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 0 01:49:07,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=2, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:07,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 2 01:49:07,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 0 01:49:07,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from -1 to 0 01:49:07,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62} 01:49:07,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:49:07,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionChainWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} does not match any client, pre-disabling client. 01:49:07,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 01:49:07,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=173}] 01:49:07,38 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=0 01:49:07,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 2, nextIndex: 3 01:49:07,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 01:49:07,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 1 01:49:07,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 0 to 1 01:49:07,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64} 01:49:07,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} has no callback 01:49:07,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 2 01:49:07,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 1 to 2 01:49:07,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 2 data Simple{size=173} 01:49:07,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Resuming commit of transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:49:07,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: commit - successfulFromPrevious: [] 01:49:07,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:49:07,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 01:49:07,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithSingleShard_member-1-shard-cars-testTransactionChainWithSingleShard$a] 01:49:07,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionChainWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-392600545] with name member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher 01:49:07,38 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-392600545] 01:49:07,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 1 for term 3 01:49:07,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:49:07,38 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 01:49:07,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=2 01:49:07,39 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 103.1 μs 01:49:07,39 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-392600545] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:07,39 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransactionChainProxy - Tx: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 - previous Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 readied - proceeding to FindPrimaryShard 01:49:07,39 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 modifications completed with CommitTransactionReply [version=13] 01:49:07,39 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] for shard cars 01:49:07,39 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] found - trying create transaction 01:49:07,39 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] 01:49:07,39 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 2 01:49:07,39 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 01:49:07,39 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 1 01:49:07,39 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 0 to 1 01:49:07,39 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64} 01:49:07,39 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 2 01:49:07,39 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 1 to 2 01:49:07,39 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 2 data Simple{size=173} 01:49:07,40 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithSingleShard: Applying foreign transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:49:07,40 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionChainWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-400574681] with name member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher 01:49:07,40 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 1 for term 3 01:49:07,40 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-400574681] 01:49:07,40 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=2 01:49:07,40 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 01:49:07,40 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-400574681] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:07,40 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 76.71 μs 01:49:07,41 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Creating transaction : member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 01:49:07,41 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} 01:49:07,41 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@1a38e01f 01:49:07,41 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_27#-341551375] 01:49:07,42 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_27#-341551375] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:07,42 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_27#-341551375, transactionId=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0, version=13] 01:49:07,43 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:49:07,43 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_27#-341551375)] 01:49:07,44 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 ReadData operation succeeded 01:49:07,44 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:49:07,44 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 01:49:07,44 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:49:07,44 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] found - trying create transaction 01:49:07,44 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] 01:49:07,44 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Queuing TransactionOperation 01:49:07,45 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Creating transaction : member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:49:07,45 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} 01:49:07,45 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@1a38e01f 01:49:07,45 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_28#1271636726] 01:49:07,45 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_28#1271636726] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:07,46 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_28#1271636726, transactionId=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, version=13] 01:49:07,46 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:49:07,46 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_28#1271636726)] 01:49:07,47 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 ReadData operation succeeded 01:49:07,47 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:49:07,47 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 01:49:07,47 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeRead DataExists called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 01:49:07,47 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 sending 2 batched modifications, ready: false 01:49:07,47 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_28#1271636726)] 01:49:07,48 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.DataExists to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_28#1271636726)] 01:49:07,50 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 modifications completed with BatchedModificationsReply [numBatched=2] 01:49:07,50 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 DataExists operation succeeded 01:49:07,50 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:49:07,50 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_28#1271636726)] 01:49:07,51 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 ReadData operation succeeded 01:49:07,51 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 01:49:07,51 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeDelete called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 01:49:07,51 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 canCommit 01:49:07,51 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 successfully completed direct commit 01:49:07,52 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Readying 1 components for commit 01:49:07,52 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Readying transaction for shard cars 01:49:07,52 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 directCommit called 01:49:07,52 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 sending 2 batched modifications, ready: true 01:49:07,52 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_28#1271636726)] 01:49:07,52 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 canCommit 01:49:07,54 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:49:07,54 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: handleForwardedReadyTransaction for member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:49:07,54 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, client version 13 01:49:07,54 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:49:07,54 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 ready 01:49:07,54 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=MERGE, modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}], operation=MERGE}}}}}}}]} 01:49:07,54 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Validating transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:49:07,54 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 validated 01:49:07,54 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:07,54 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: canCommit for member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: success 01:49:07,54 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Committing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:49:07,54 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Preparing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:49:07,54 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 candidate ready 01:49:07,54 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: canCommit - candidate rootPath: / 01:49:07,54 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: canCommit - messages: [] 01:49:07,54 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: preCommit - successfulFromPrevious: [] 01:49:07,55 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 prepared 01:49:07,55 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:07,55 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Finishing commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:49:07,55 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Starting commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:49:07,55 PM [cluster-test-shard-dispatcher-15] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 229 01:49:07,55 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=229}] 01:49:07,55 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=229}] 01:49:07,55 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:07,55 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 01:49:07,55 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 01:49:07,55 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=2, prevLogTerm=3, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=229}]]] 01:49:07,55 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 submitted to persistence 01:49:07,55 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=229}] 01:49:07,56 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=2, prevLogTerm=3, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=229}]]] 01:49:07,56 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 1 01:49:07,56 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:49:07,56 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log Simple{size=229} 01:49:07,56 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=229}] 01:49:07,56 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 2 01:49:07,56 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:07,56 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=229}] 01:49:07,56 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 3, nextIndex: 4 01:49:07,56 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 01:49:07,56 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 3 01:49:07,56 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 2 to 3 01:49:07,56 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 3 data Simple{size=229} 01:49:07,56 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Resuming commit of transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:49:07,57 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: commit - successfulFromPrevious: [] 01:49:07,57 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 01:49:07,57 PM [cluster-test-shard-dispatcher-15] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=MERGE, modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}], operation=MERGE, modificationType=WRITE}}}}}}}]} 01:49:07,57 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithSingleShard_member-1-shard-cars-testTransactionChainWithSingleShard_shard-cars-member-2%3Adatastore-testTransactionChainWithSingleShard%400%3A1-2_28%231271636726$i] 01:49:07,57 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 01:49:07,57 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 2 for term 3 01:49:07,57 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:49:07,57 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 106.2 μs 01:49:07,57 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=3 01:49:07,58 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 modifications completed with CommitTransactionReply [version=13] 01:49:07,58 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 successfully completed direct commit 01:49:07,58 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 3 01:49:07,58 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 01:49:07,58 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 3 01:49:07,58 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 2 to 3 01:49:07,58 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 3 data Simple{size=229} 01:49:07,58 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 01:49:07,58 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithSingleShard: Applying foreign transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:49:07,58 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 01:49:07,58 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:49:07,58 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] found - trying create transaction 01:49:07,58 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] 01:49:07,58 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:49:07,59 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 2 for term 3 01:49:07,59 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=3 01:49:07,59 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 01:49:07,59 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 103.6 μs 01:49:07,63 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} 01:49:07,63 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Creating transaction : member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 01:49:07,64 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_29#-2086243694] 01:49:07,64 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_29#-2086243694] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:07,64 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_29#-2086243694, transactionId=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0, version=13] 01:49:07,64 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:49:07,64 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_29#-2086243694)] 01:49:07,66 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 ReadData operation succeeded 01:49:07,66 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:49:07,66 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testTransactionChainWithSingleShard 01:49:07,66 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithSingleShard - waiting on 1 shards 01:49:07,66 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onShutDown 01:49:07,66 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Initiating leader transfer 01:49:07,66 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@29b37a61 01:49:07,66 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@29b37a61 01:49:07,66 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Attempting to transfer leadership 01:49:07,66 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:49:07,66 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:49:07,67 PM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:07,67 PM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:07,68 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:49:07,69 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard] 01:49:07,70 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionChainWithSingleShard, matchIndex: 3, lastIndex: 3, isVoting: true 01:49:07,70 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 01:49:07,70 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:49:07,70 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Leader transfer complete 01:49:07,74 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:49:07,74 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:07,74 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:07,481 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: All shards shutdown - sending PoisonPill to self 01:49:07,481 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] ShardManager - shard-manager-testTransactionChainWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#858296809]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:49:07,482 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionChainWithSingleShard 01:49:07,482 PM [cluster-test-shard-dispatcher-15] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard (Leader)_slicer-id-525: Closing 01:49:07,482 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Aborting 0 pending queued transactions 01:49:07,482 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithSingleShard 01:49:07,482 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 01:49:07,482 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:49:07,482 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: shutting down 01:49:07,482 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 01:49:07,483 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 01:49:07,483 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#468849845] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#468849845] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:07,483 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#468849845] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#468849845] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:07,484 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:07,484 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:07,492 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:07,492 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:07,492 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:07,492 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:07,493 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:07,503 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithSingleShard: Closing 01:49:07,503 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionChainWithSingleShard 01:49:07,503 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Closing 01:49:07,503 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Aborting 0 pending queued transactions 01:49:07,504 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithSingleShard 01:49:07,504 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:07,504 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:08,507 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [340122778932299608] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:08,517 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-9049490931874339599] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:08,524 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:08,524 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:49:08.534] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:08,544 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:08,544 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:08,547 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:08.547] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:08,564 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:08,574 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [7212200058422749109] 01:49:08,574 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:08,576 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:08,576 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:08,583 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:08,583 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:08,587 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:08,587 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:08,587 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:08,589 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:08,596 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-6938507727424882885] 01:49:08,596 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:08,605 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 closeTransaction called 01:49:08,605 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_29#-2086243694)] 01:49:08,605 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 closeTransaction called 01:49:08,605 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_27#-341551375)] 01:49:08,606 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:08,606 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:08,611 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:08,611 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:08,619 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:08,638 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-3450214933205057173] 01:49:08,638 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:08,639 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:08,644 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:08,644 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:08,644 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:08,644 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:08,650 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:08,650 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:08,652 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:08,653 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:08,653 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:08,655 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:08,664 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 01:49:08,665 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 01:49:08,673 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:08,673 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:09,689 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:09.690] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:09,699 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:09,700 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:09,701 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:09.702] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:09,715 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:09,721 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-4613166625906947615] 01:49:09,721 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:09,723 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:09,723 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:09,732 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:09,732 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:09,732 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:09,732 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:09,732 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:09,737 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:09,746 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-4526564204648466160] 01:49:09,746 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:09,748 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:09,748 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:09,757 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:09,757 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:09,828 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:09,858 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [5389783218566935170] 01:49:09,858 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:09,860 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:09,860 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:09,863 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:09,863 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:09,867 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:09,868 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:09,869 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:09,870 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:09,871 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testForwardedReadyTransactionForwardedToLeader 01:49:09,875 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1905709317] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:09,875 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 01:49:09,884 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: no initial generation override, starting from 0 01:49:09,884 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 01:49:09,884 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@115e9634 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@29b03305 01:49:09,884 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2d8fa2eb 01:49:09,884 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, 0, 1710164949884, None)) 01:49:09,884 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164949883,0,0)) 01:49:09,884 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@29b03305 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6d4a3367 01:49:09,884 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testForwardedReadyTransactionForwardedToLeader 01:49:09,884 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Creating local shard: member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:09,885 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} started 01:49:09,885 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:09,885 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:09,885 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@672f8761 01:49:09,885 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:09,885 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:09,885 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:09,887 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:09,889 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:09,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:09,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: updated peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-683327897] 01:49:09,889 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testForwardedReadyTransactionForwardedToLeader 01:49:09,889 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, persistent : true 01:49:09,890 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-683327897] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:09,890 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#1359827435 created and ready for shard:member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:09,890 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#1359827435] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:09,890 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-1:snapshot-read#414584640] 01:49:09,890 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-1:snapshot-read#414584640] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:09,894 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1773390547] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:09,911 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with journal batch size 1 01:49:09,912 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:09,912 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:09,913 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Initializing shard [cars] 01:49:09,913 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is initialized 01:49:09,913 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:49:09,913 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 01:49:09,913 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from null to Follower 01:49:09,913 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader 01:49:09,913 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from null to Follower 01:49:09,924 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 01:49:09,930 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:49:09,930 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:09,936 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: no initial generation override, starting from 0 01:49:09,936 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 01:49:09,937 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1c4e2527 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@20d0de61 01:49:09,937 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1efa81eb 01:49:09,937 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, 0, 1710164949937, None)) 01:49:09,937 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testForwardedReadyTransactionForwardedToLeader 01:49:09,937 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Creating local shard: member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:09,937 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:09,937 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:09,937 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164949936,0,0)) 01:49:09,937 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@20d0de61 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@313e3921 01:49:09,938 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4471bb8d 01:49:09,938 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} started 01:49:09,938 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:09,938 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:09,938 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:09,938 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:09,938 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: updated peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1398553441] 01:49:09,939 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, persistent : true 01:49:09,939 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with journal batch size 1 01:49:09,939 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1398553441] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:09,939 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-2:snapshot-read#-213328891] 01:49:09,940 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-1734678136 created and ready for shard:member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:09,940 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-2:snapshot-read#-213328891] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:09,940 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:09,940 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:09,940 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:49:09,940 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 01:49:09,940 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Initializing shard [cars] 01:49:09,940 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader is initialized 01:49:09,940 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:09,940 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from null to Follower 01:49:09,940 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader 01:49:09,940 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader from null to Follower 01:49:09,942 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-1734678136] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:09,988 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1905709317] 01:49:09,989 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 01:49:09,989 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-683327897] 01:49:10,212 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:10,212 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Received ElectionTimeout - switching to Candidate 01:49:10,212 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] 01:49:10,212 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:10,212 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 1 01:49:10,212 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:10,212 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Closing 01:49:10,212 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:49:10,212 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 01:49:10,212 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from Follower to Candidate 01:49:10,213 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from Follower to Candidate 01:49:10,442 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:49:10,442 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:10,442 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 2 01:49:10,543 PM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:10,692 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:49:10,693 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:10,693 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 3 01:49:10,743 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:49:10,743 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:10,747 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:10,747 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:10,747 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:10,747 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:10,747 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:10,747 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:10,747 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:10,752 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:10,752 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:10,752 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:10,752 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:10,752 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:10,752 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:10,873 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:49:10,982 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:49:10,982 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:10,983 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 4 01:49:10,983 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:10,984 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:10,984 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Cluster state: [] 01:49:10,984 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:10,984 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:10,984 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=null 01:49:10,984 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:10,984 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:10,984 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:49:10,987 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:49:10,987 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:49:10,987 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] 01:49:10,987 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:10,987 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:49:10,987 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:49:10,987 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: became leader with frontend state for [] 01:49:10,987 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from Candidate to Leader 01:49:10,987 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderPayloadVersion=13] 01:49:10,987 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:10,987 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:10,987 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from Candidate to Leader 01:49:10,987 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All Shards are ready - data store testForwardedReadyTransactionForwardedToLeader is ready 01:49:10,988 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Closing 01:49:10,988 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:10,988 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderPayloadVersion=13] 01:49:10,988 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:10,988 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:10,988 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All Shards are ready - data store testForwardedReadyTransactionForwardedToLeader is ready 01:49:10,988 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader status sync done false 01:49:10,989 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: -1, nextIndex: 0 01:49:10,989 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:11,52 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1773390547] 01:49:11,52 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 01:49:11,52 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1398553441] 01:49:11,52 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:11,52 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1773390547] 01:49:11,52 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 01:49:11,52 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1398553441] 01:49:11,64 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: handleForwardedReadyTransaction for member-1-ShardTransactionTest-fe-0-txn-227-0 01:49:11,64 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:11,64 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader)] 01:49:11,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, disabling transaction tracking 01:49:11,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:49:11,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:49:11,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:49:11,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:11,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:11,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 01:49:11,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying 3 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-227-0 01:49:11,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-227-0 of 3 operations, client version 13 01:49:11,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-227-0 01:49:11,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-227-0 ready 01:49:11,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-227-0 01:49:11,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 validated 01:49:11,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:11,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-227-0: success 01:49:11,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-227-0 01:49:11,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-227-0 01:49:11,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 candidate ready 01:49:11,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-227-0: canCommit - candidate rootPath: / 01:49:11,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-227-0: canCommit - messages: [] 01:49:11,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-227-0: preCommit - successfulFromPrevious: [] 01:49:11,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 prepared 01:49:11,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:11,68 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-227-0 01:49:11,68 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-227-0 01:49:11,68 PM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 197 01:49:11,68 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:49:11,68 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:49:11,68 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-227-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:11,68 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:11,68 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:11,68 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 submitted to persistence 01:49:11,69 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:49:11,69 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:49:11,69 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 01:49:11,69 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:49:11,69 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:49:11,69 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:49:11,69 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:49:11,69 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 1 01:49:11,69 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:11,70 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 0, nextIndex: 1 01:49:11,70 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:49:11,70 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 0 01:49:11,70 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from -1 to 0 01:49:11,70 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:49:11,71 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 01:49:11,71 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 01:49:11,71 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, nanosAgo=4172028} is already disabled 01:49:11,71 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:11,71 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:11,71 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 01:49:11,71 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 01:49:11,71 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:49:11,71 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader status sync done true 01:49:11,73 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 01:49:11,73 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:49:11,73 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:49:11,73 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log Simple{size=197} 01:49:11,73 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:49:11,73 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 2 01:49:11,73 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 0 01:49:11,73 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:11,73 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:49:11,73 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 0 01:49:11,73 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from -1 to 0 01:49:11,73 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:49:11,73 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 01:49:11,73 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 01:49:11,74 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 1, nextIndex: 2 01:49:11,74 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:49:11,74 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 1 01:49:11,74 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 0 to 1 01:49:11,74 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 01:49:11,74 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:49:11,74 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-227-0 01:49:11,74 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-227-0: commit - successfulFromPrevious: [] 01:49:11,74 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 changing state from COMMIT_PENDING to COMMITTED 01:49:11,74 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/system/testProbe-762#1742664546] 01:49:11,74 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 01:49:11,74 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-987460966] with name member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher 01:49:11,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 01:49:11,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:11,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 01:49:11,75 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-987460966] 01:49:11,75 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 01:49:11,75 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 74.78 μs 01:49:11,75 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-987460966] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:11,75 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 1 01:49:11,75 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 01:49:11,75 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:49:11,75 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 1 01:49:11,75 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 0 to 1 01:49:11,75 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 01:49:11,75 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1905709317] 01:49:11,75 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 Queuing TransactionOperation 01:49:11,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:11,75 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-227-0 01:49:11,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-683327897, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4a7ec930] 01:49:11,76 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-683327897, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4a7ec930] 01:49:11,76 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-683327897)] for shard cars 01:49:11,76 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 01:49:11,76 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@3871efa6 01:49:11,76 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-1759321785] with name member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher 01:49:11,76 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 01:49:11,76 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 01:49:11,76 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-1759321785] 01:49:11,76 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 01:49:11,76 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 94.80 μs 01:49:11,76 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-1759321785] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:11,76 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: handleForwardedReadyTransaction for member-1-ShardTransactionTest-fe-0-txn-228-0 01:49:11,77 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:11,77 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader)] 01:49:11,78 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-228-0 01:49:11,78 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-228-0 of 1 operations, client version 13 01:49:11,78 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-228-0 01:49:11,78 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-228-0 ready 01:49:11,79 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0 canCommit 01:49:11,80 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-683327897)] resolved 01:49:11,80 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: successfully resolved all cohort actors 01:49:11,80 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0 finishCanCommit 01:49:11,80 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: sending CanCommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-228-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-683327897)] 01:49:11,80 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-683327897)] 01:49:11,80 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Can committing transaction member-1-ShardTransactionTest-fe-0-txn-228-0 01:49:11,80 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-228-0 01:49:11,80 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 validated 01:49:11,80 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:11,80 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-228-0: success 01:49:11,80 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:49:11,80 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: canCommit returning result: true 01:49:11,80 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0 commit 01:49:11,80 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0 finish commit 01:49:11,80 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: Sending CommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-228-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-683327897)] 01:49:11,80 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-683327897)] 01:49:11,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-228-0 01:49:11,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-228-0 01:49:11,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 candidate ready 01:49:11,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-228-0: canCommit - candidate rootPath: / 01:49:11,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-228-0: canCommit - messages: [] 01:49:11,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-228-0: preCommit - successfulFromPrevious: [] 01:49:11,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 prepared 01:49:11,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:11,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-228-0 01:49:11,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-228-0 01:49:11,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 208 01:49:11,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:49:11,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:49:11,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-228-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:11,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:49:11,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:11,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 01:49:11,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 submitted to persistence 01:49:11,82 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:49:11,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 01:49:11,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:49:11,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:49:11,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log Simple{size=208} 01:49:11,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:49:11,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 2 01:49:11,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:11,83 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:49:11,83 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 2, nextIndex: 3 01:49:11,83 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:49:11,83 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 2 01:49:11,83 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 1 to 2 01:49:11,83 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 01:49:11,83 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-228-0 01:49:11,83 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-228-0: commit - successfulFromPrevious: [] 01:49:11,83 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 changing state from COMMIT_PENDING to COMMITTED 01:49:11,83 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testForwardedReadyTransactionForwardedToLeader_member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader%23-683327897$a] 01:49:11,83 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: commit succeeded 01:49:11,83 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 01:49:11,83 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 01:49:11,83 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:49:11,83 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 01:49:11,83 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 88.65 μs 01:49:11,84 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0 01:49:11,84 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-683327897)] for shard cars, trying to use DirectTransactionContextWrapper 01:49:11,84 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@3871efa6 01:49:11,84 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 01:49:11,84 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 01:49:11,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Issuing gracefulStop to shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:11,84 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader - waiting on 1 shards 01:49:11,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onShutDown 01:49:11,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Initiating leader transfer 01:49:11,85 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@158ae0ad 01:49:11,85 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@158ae0ad 01:49:11,85 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Attempting to transfer leadership 01:49:11,85 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 01:49:11,85 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:49:11,85 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:11,85 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:11,86 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 2 01:49:11,86 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:49:11,87 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 2 01:49:11,87 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 1 to 2 01:49:11,87 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 01:49:11,87 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-228-0 01:49:11,87 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 01:49:11,87 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 01:49:11,87 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 01:49:11,87 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader] 01:49:11,87 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 90.11 μs 01:49:11,87 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 01:49:11,87 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:11,87 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:11,89 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, matchIndex: 2, lastIndex: 2, isVoting: true 01:49:11,89 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Follower's log matches - sending ElectionTimeout 01:49:11,89 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:49:11,89 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Leader transfer complete 01:49:11,502 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 01:49:11,502 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-683327897]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:49:11,503 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:11,503 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader)_slicer-id-530: Closing 01:49:11,503 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Aborting 0 pending queued transactions 01:49:11,503 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 01:49:11,503 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 01:49:11,504 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 01:49:11,504 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: shutting down 01:49:11,504 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 01:49:11,504 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1905709317] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1905709317] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:11,504 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1905709317] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1905709317] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:11,504 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 01:49:11,504 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1905709317] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1905709317] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:11,504 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#1138131275] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1138131275] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:11,504 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#1138131275] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1138131275] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:11,504 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:11,504 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:11,510 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:11,510 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:11,510 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:11,510 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:11,512 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:11,523 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 01:49:11,523 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:49:11,523 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Closing 01:49:11,523 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Aborting 0 pending queued transactions 01:49:11,524 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 01:49:11,524 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:11,524 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:12,523 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-4613166625906947615] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:12,543 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-4526564204648466160] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:12,545 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:12,545 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:12,547 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:12.548] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:12,557 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:12,557 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:12,560 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:12.562] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:12,592 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:49:12,599 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [414254056503579655] 01:49:12,599 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:12,601 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:12,601 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:12,601 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:12,607 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:12,611 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:12,611 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:12,611 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:12,614 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:12,618 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-1461512540701404034] 01:49:12,619 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:12,620 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:12,620 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:12,630 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:12,630 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:12,640 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:12,649 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-209487583186554927] 01:49:12,651 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:12,652 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:12,652 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:12,662 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:12,662 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:12,662 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:12,666 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:12,667 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithSingleShard 01:49:12,670 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:12,677 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:12,678 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-1949139804] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:12,679 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 01:49:12,684 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: no initial generation override, starting from 0 01:49:12,684 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:49:12,684 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7ad8962 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@76bf3e62 01:49:12,684 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@734d2dea 01:49:12,684 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, 0, 1710164952684, None)) 01:49:12,684 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164952683,0,0)) 01:49:12,684 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@76bf3e62 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6dbbf03e 01:49:12,684 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} started 01:49:12,685 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:12,686 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:12,687 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithSingleShard 01:49:12,690 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 01:49:12,692 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-525410118] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:12,696 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: no initial generation override, starting from 0 01:49:12,696 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:49:12,696 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithSingleShard 01:49:12,696 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@11033e13 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@509c0554 01:49:12,696 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Creating local shard: member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:12,696 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4f7e450c 01:49:12,696 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, 0, 1710164952696, None)) 01:49:12,696 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:12,696 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:12,696 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164952695,0,0)) 01:49:12,697 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@e4c9b95 01:49:12,697 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@509c0554 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3c396767 01:49:12,697 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} started 01:49:12,697 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:12,697 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:12,697 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:12,697 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:12,697 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: updated peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard on shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard#569224982] 01:49:12,697 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testChainedTransactionFailureWithSingleShard, persistent : true 01:49:12,698 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard#569224982] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:12,698 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-2:snapshot-read#-612244794] 01:49:12,698 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-407026826 created and ready for shard:member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:12,698 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-2:snapshot-read#-612244794] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:12,698 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-407026826] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:12,698 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testChainedTransactionFailureWithSingleShard with journal batch size 1 01:49:12,698 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:12,698 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:12,699 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:49:12,699 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 01:49:12,699 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:12,699 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Initializing shard [cars] 01:49:12,699 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard is initialized 01:49:12,699 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from null to Follower 01:49:12,699 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard 01:49:12,699 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-2-shard-cars-testChainedTransactionFailureWithSingleShard from null to Follower 01:49:12,701 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:49:12,701 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithSingleShard 01:49:12,701 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Creating local shard: member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:12,701 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:12,701 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:12,701 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@232b99ac 01:49:12,701 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:12,701 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:12,702 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:12,702 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:12,702 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: updated peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard on shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-1818233065] 01:49:12,702 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-1-shard-cars-testChainedTransactionFailureWithSingleShard, persistent : true 01:49:12,702 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-1818233065] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:12,702 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-1-shard-cars-testChainedTransactionFailureWithSingleShard with journal batch size 1 01:49:12,702 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#802107646 created and ready for shard:member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:12,702 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#802107646] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:12,705 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:12,706 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:12,707 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:12,707 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Initializing shard [cars] 01:49:12,707 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is initialized 01:49:12,707 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:49:12,707 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 01:49:12,707 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from null to Follower 01:49:12,707 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard 01:49:12,707 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from null to Follower 01:49:12,708 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-1:snapshot-read#42705366] 01:49:12,708 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-1:snapshot-read#42705366] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:12,747 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-1949139804] 01:49:12,747 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: findLocalShard : cars 01:49:12,747 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-1818233065] 01:49:12,989 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:12,989 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 01:49:12,989 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testChainedTransactionFailureWithSingleShard] 01:49:12,989 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:12,989 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 1 01:49:12,989 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:12,989 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Closing 01:49:12,989 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:49:12,989 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 01:49:12,989 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from Follower to Candidate 01:49:12,989 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from Follower to Candidate 01:49:13,228 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 01:49:13,228 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:13,228 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 2 01:49:13,283 PM [cluster-test-shard-dispatcher-23] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:13,498 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 01:49:13,498 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:13,499 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 3 01:49:13,620 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:13,620 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:13,620 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:13,620 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:13,620 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:13,620 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:13,620 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:13,620 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:49:13,620 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:13,626 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:13,626 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:13,626 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:13,626 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:13,626 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:13,626 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:13,681 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:49:13,738 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 01:49:13,738 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:13,738 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 4 01:49:13,739 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:13,743 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:13,743 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Cluster state: [] 01:49:13,743 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:13,743 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:13,743 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=null 01:49:13,743 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:13,743 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:13,743 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:49:13,745 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:49:13,745 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:49:13,745 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testChainedTransactionFailureWithSingleShard] 01:49:13,745 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:13,745 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:49:13,745 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:49:13,745 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: became leader with frontend state for [] 01:49:13,745 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderPayloadVersion=13] 01:49:13,745 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:13,745 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:13,745 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from Candidate to Leader 01:49:13,746 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from Candidate to Leader 01:49:13,746 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All Shards are ready - data store testChainedTransactionFailureWithSingleShard is ready 01:49:13,747 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Closing 01:49:13,747 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testChainedTransactionFailureWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:13,747 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderPayloadVersion=13] 01:49:13,747 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:13,747 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:13,747 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All Shards are ready - data store testChainedTransactionFailureWithSingleShard is ready 01:49:13,747 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithSingleShard status sync done false 01:49:13,748 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: -1, nextIndex: 0 01:49:13,748 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:13,755 PM [main] [DEBUG] AbstractDOMBroker - Transaction chain 0 created with listener Mock for DOMTransactionChainListener, hashCode: 2025868403, backing store chains {CONFIGURATION=org.opendaylight.controller.cluster.datastore.TransactionChainProxy@3bfb01cd} 01:49:13,756 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testChainedTransactionFailureWithSingleShard commit rateLimit adjusted to 95.19145652890465 01:49:13,756 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 01:49:13,756 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-525410118] 01:49:13,756 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:49:13,757 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Readying 1 components for commit 01:49:13,757 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 01:49:13,757 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:49:13,757 PM [main] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 is submitted for execution. 01:49:13,757 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 canCommit 01:49:13,757 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:13,757 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard, primaryVersion=13] 01:49:13,757 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard, primaryVersion=13] 01:49:13,757 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard)] for shard cars 01:49:13,757 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard)] found - creating WRITE_ONLY transaction context 01:49:13,757 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:49:13,757 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 directCommit called 01:49:13,757 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 sending 1 batched modifications, ready: true 01:49:13,757 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard)] 01:49:13,759 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, disabling transaction tracking 01:49:13,759 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 01:49:13,759 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 01:49:13,759 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:49:13,759 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:13,759 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:13,759 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}]]] 01:49:13,759 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 01:49:13,759 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 01:49:13,759 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:49:13,759 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0} 01:49:13,759 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@582eb00f 01:49:13,759 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying 1 batched modifications for Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 01:49:13,760 PM [cluster-test-shard-dispatcher-27] [ERROR] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Error handling BatchedModifications for Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:49:13,761 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 01:49:13,761 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 01:49:13,761 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}]]] 01:49:13,761 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Number of entries to be appended = 1 01:49:13,761 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:49:13,761 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71} 01:49:13,761 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 01:49:13,761 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Log size is now 1 01:49:13,761 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:13,762 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 01:49:13,762 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithSingleShard status sync done true 01:49:13,762 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: 0, nextIndex: 1 01:49:13,762 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:49:13,762 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Setting last applied to 0 01:49:13,762 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from -1 to 0 01:49:13,762 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71} 01:49:13,763 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:49:13,763 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} does not match any client, pre-disabling client. 01:49:13,763 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, nanosAgo=3831573} is already disabled 01:49:13,763 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:13,763 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:13,763 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}]]] 01:49:13,763 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=0 01:49:13,764 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}]]] 01:49:13,764 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during phase CAN_COMMIT, starting Abort org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:49:13,764 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 modifications failed org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:49:13,764 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DOMBrokerTransactionChain - Transaction chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@17b5b08d failed. TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema.]]} at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1119) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) at org.opendaylight.controller.cluster.datastore.SingleCommitCohortProxy$1.onComplete(SingleCommitCohortProxy.java:60) at akka.dispatch.OnComplete.internal(Future.scala:299) at akka.dispatch.OnComplete.internal(Future.scala:297) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:49:13,764 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Number of entries to be appended = 1 01:49:13,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:49:13,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73} 01:49:13,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 01:49:13,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Log size is now 2 01:49:13,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Commit index set to 0 01:49:13,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:13,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:49:13,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Setting last applied to 0 01:49:13,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from -1 to 0 01:49:13,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71} 01:49:13,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:49:13,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} does not match any client, pre-disabling client. 01:49:13,765 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 aborted successfully 01:49:13,766 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 01:49:13,766 PM [main] [DEBUG] DOMBrokerTransactionChain - Chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@17b5b08d is no longer running 01:49:13,766 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=0 01:49:13,766 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:49:13,766 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:13,766 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithSingleShard - waiting on 1 shards 01:49:13,766 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onShutDown 01:49:13,766 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Initiating leader transfer 01:49:13,766 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2bd306df 01:49:13,766 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2bd306df 01:49:13,766 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Attempting to transfer leadership 01:49:13,766 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:13,766 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:13,767 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:49:13,767 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:13,767 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:13,768 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: 1, nextIndex: 2 01:49:13,768 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:49:13,768 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Setting last applied to 1 01:49:13,768 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from 0 to 1 01:49:13,768 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73} 01:49:13,768 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0} has no callback 01:49:13,768 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: fakeSnapshot purging log to 0 for term 4 01:49:13,768 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:13,768 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testChainedTransactionFailureWithSingleShard, matchIndex: 1, lastIndex: 1, isVoting: true 01:49:13,768 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 01:49:13,769 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:13,769 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Leader transfer complete 01:49:13,769 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=1 01:49:13,769 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard] 01:49:13,769 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:49:13,769 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:13,769 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:13,769 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Commit index set to 1 01:49:13,769 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:49:13,769 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Setting last applied to 1 01:49:13,769 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from 0 to 1 01:49:13,769 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73} 01:49:13,769 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: fakeSnapshot purging log to 0 for term 4 01:49:13,770 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=1 01:49:13,980 PM [cluster-test-shard-dispatcher-27] [WARN] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 210, lastApplied : 1, commitIndex : 1 01:49:14,178 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All shards shutdown - sending PoisonPill to self 01:49:14,178 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-1818233065]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:49:14,178 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:14,178 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader)_slicer-id-535: Closing 01:49:14,178 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Aborting 0 pending queued transactions 01:49:14,179 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 01:49:14,179 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 01:49:14,179 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:49:14,179 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 01:49:14,179 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: shutting down 01:49:14,180 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 01:49:14,180 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:14,180 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:14,185 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:14,185 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:14,186 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:14,186 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:14,188 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:14,199 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 01:49:14,199 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:49:14,199 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Closing 01:49:14,199 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Aborting 0 pending queued transactions 01:49:14,199 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 01:49:14,199 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:14,199 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:15,200 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [414254056503579655] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:15,215 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:15,215 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:15,217 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:15.218] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:15,220 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-1461512540701404034] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:15,226 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:15,226 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:15,228 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:15.229] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:15,251 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:15,258 PM [cluster-test-akka.actor.default-dispatcher-9] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [5308539937432240015] 01:49:15,263 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:15,263 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:15,263 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:15,263 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:15,263 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:15,270 PM [cluster-test-akka.actor.default-dispatcher-9] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:15,270 PM [cluster-test-akka.actor.default-dispatcher-9] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:15,270 PM [cluster-test-akka.actor.default-dispatcher-9] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:15,274 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:15,281 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4318444711393860697] 01:49:15,281 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:15,282 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:15,282 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:15,283 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:15,283 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:15,311 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:15,320 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [6324602531606998368] 01:49:15,321 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:15,322 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:15,322 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:15,330 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:15,332 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:15,333 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithShardLeaderNotResponding 01:49:15,335 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:15,335 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:15,337 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#1753453330] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:15,338 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 01:49:15,341 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:15,342 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithShardLeaderNotResponding 01:49:15,342 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Creating local shard: member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:15,342 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:15,342 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:15,343 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:15,358 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: no initial generation override, starting from 0 01:49:15,358 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:49:15,358 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@415fc74b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@67458c94 01:49:15,358 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@22470e18 01:49:15,358 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, 0, 1710164955358, None)) 01:49:15,359 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164955357,0,0)) 01:49:15,359 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@67458c94 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@58451389 01:49:15,359 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5552fd93 01:49:15,359 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} started 01:49:15,360 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:15,362 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:15,364 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithShardLeaderNotResponding 01:49:15,365 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:15,365 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:15,367 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#-1471262568] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:15,379 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 01:49:15,379 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:15,379 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:15,379 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: updated peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding on shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1715205138] 01:49:15,380 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithShardLeaderNotResponding, persistent : true 01:49:15,380 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1715205138] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:15,381 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-606329481 created and ready for shard:member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:15,381 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-1:snapshot-read#-2070244366] 01:49:15,381 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-606329481] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:15,381 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-1:snapshot-read#-2070244366] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:15,381 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: no initial generation override, starting from 0 01:49:15,381 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:49:15,381 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@412ca1b9 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@42356262 01:49:15,381 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithShardLeaderNotResponding with journal batch size 1 01:49:15,381 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4a4a3335 01:49:15,381 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithShardLeaderNotResponding 01:49:15,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Creating local shard: member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:15,382 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:15,382 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:15,382 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, 0, 1710164955381, None)) 01:49:15,384 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:15,384 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:15,384 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 01:49:15,384 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 01:49:15,384 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Initializing shard [cars] 01:49:15,384 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is initialized 01:49:15,384 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from null to Follower 01:49:15,384 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , registered listener akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding 01:49:15,384 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164955380,0,0)) 01:49:15,384 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@42356262 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6b4d05f3 01:49:15,384 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from null to Follower 01:49:15,386 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} started 01:49:15,387 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6fe9be2c 01:49:15,387 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:15,387 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:15,387 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:15,387 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:15,387 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: updated peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding on shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding#132059175] 01:49:15,387 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithShardLeaderNotResponding, persistent : true 01:49:15,388 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithShardLeaderNotResponding with journal batch size 1 01:49:15,388 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding#132059175] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:15,389 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-1241871629 created and ready for shard:member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:15,389 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-1241871629] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:15,389 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-2:snapshot-read#-2071255974] 01:49:15,390 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:15,390 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:15,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Initializing shard [cars] 01:49:15,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding is initialized 01:49:15,390 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 01:49:15,390 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 01:49:15,390 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:15,390 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from null to Follower 01:49:15,390 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithShardLeaderNotResponding , registered listener akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding 01:49:15,390 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-2-shard-cars-testTransactionWithShardLeaderNotResponding from null to Follower 01:49:15,391 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-2:snapshot-read#-2071255974] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:15,392 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:49:15,396 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:15,437 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#1753453330] 01:49:15,437 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: findLocalShard : cars 01:49:15,437 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1715205138] 01:49:15,637 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:15,637 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Received ElectionTimeout - switching to Candidate 01:49:15,637 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithShardLeaderNotResponding] 01:49:15,637 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:15,637 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 1 01:49:15,637 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:15,637 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Closing 01:49:15,637 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 01:49:15,637 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 01:49:15,637 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from Follower to Candidate 01:49:15,637 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from Follower to Candidate 01:49:15,936 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 01:49:15,936 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:15,936 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 2 01:49:16,247 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 01:49:16,247 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:16,247 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 3 01:49:16,277 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:49:16,277 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:16,277 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:16,277 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:16,277 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:16,277 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:16,277 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:16,277 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:16,278 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:16,286 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:16,286 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:16,286 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:16,286 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:16,286 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:16,286 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:16,336 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:49:16,496 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 01:49:16,496 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:16,496 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 4 01:49:16,496 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:16,498 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:16,498 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Cluster state: [] 01:49:16,498 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:16,498 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:16,498 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=null 01:49:16,498 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:16,498 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:16,498 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:49:16,499 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:49:16,499 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:49:16,499 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithShardLeaderNotResponding] 01:49:16,500 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:16,500 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:49:16,500 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: true, hasLeader : true 01:49:16,500 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: became leader with frontend state for [] 01:49:16,500 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from Candidate to Leader 01:49:16,500 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderPayloadVersion=13] 01:49:16,500 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:16,500 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:16,500 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from Candidate to Leader 01:49:16,500 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 01:49:16,501 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Closing 01:49:16,501 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithShardLeaderNotResponding peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:16,501 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithShardLeaderNotResponding, leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderPayloadVersion=13] 01:49:16,501 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:16,501 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:16,501 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 01:49:16,501 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding Received follower initial sync status for member-2-shard-cars-testTransactionWithShardLeaderNotResponding status sync done false 01:49:16,501 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithShardLeaderNotResponding updated: matchIndex: -1, nextIndex: 0 01:49:16,501 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:16,542 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 01:49:16,542 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#-1471262568] 01:49:16,542 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 Queuing TransactionOperation 01:49:16,542 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:16,542 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding, primaryVersion=13] 01:49:16,542 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding, primaryVersion=13] 01:49:16,542 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] for shard cars 01:49:16,542 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 01:49:16,542 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 01:49:16,544 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, disabling transaction tracking 01:49:16,544 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 01:49:16,544 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 01:49:16,544 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:49:16,544 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:16,544 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:16,544 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}]]] 01:49:16,544 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Creating transaction : member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 01:49:16,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_30#-1285259706] 01:49:16,545 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_30#-1285259706] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:16,545 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 01:49:16,545 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}]]] 01:49:16,545 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Number of entries to be appended = 1 01:49:16,545 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:49:16,545 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70} 01:49:16,545 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_30#-1285259706, transactionId=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0, version=13] 01:49:16,545 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 01:49:16,546 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Log size is now 1 01:49:16,546 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithShardLeaderNotResponding, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:16,546 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:49:16,546 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_30#-1285259706)] 01:49:16,546 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 01:49:16,546 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding Received follower initial sync status for member-2-shard-cars-testTransactionWithShardLeaderNotResponding status sync done true 01:49:16,549 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithShardLeaderNotResponding updated: matchIndex: 0, nextIndex: 1 01:49:16,549 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:49:16,549 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Setting last applied to 0 01:49:16,549 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Moving last applied index from -1 to 0 01:49:16,549 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70} 01:49:16,549 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:49:16,549 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} does not match any client, pre-disabling client. 01:49:16,549 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, nanosAgo=5129564} is already disabled 01:49:16,549 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:16,549 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Persisting ApplyJournalEntries with index=0 01:49:16,552 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 ReadData operation succeeded 01:49:16,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Commit index set to 0 01:49:16,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:49:16,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Setting last applied to 0 01:49:16,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Moving last applied index from -1 to 0 01:49:16,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70} 01:49:16,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:49:16,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} does not match any client, pre-disabling client. 01:49:16,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Persisting ApplyJournalEntries with index=0 01:49:16,553 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 01:49:16,553 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:16,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader)_slicer-id-540: Closing 01:49:16,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Aborting 0 pending queued transactions 01:49:16,553 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 01:49:16,554 PM [cluster-test-akka.actor.default-dispatcher-9] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:16,554 PM [cluster-test-akka.actor.default-dispatcher-9] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:16,557 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:16,557 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:16,557 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:16,557 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:16,558 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:16,567 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithShardLeaderNotResponding 01:49:16,568 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:16,568 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionWithShardLeaderNotResponding commit rateLimit adjusted to 95.19145652890465 01:49:16,568 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 01:49:16,568 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:16,568 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] for shard cars, trying to use DirectTransactionContextWrapper 01:49:16,568 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:49:16,568 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 01:49:16,568 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 01:49:16,568 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Queuing TransactionOperation 01:49:16,569 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Readying 1 components for commit 01:49:16,569 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Readying transaction for shard cars 01:49:16,569 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Queuing TransactionOperation 01:49:16,569 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 canCommit 01:49:16,632 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 01:49:16,662 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 01:49:16,662 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 01:49:16,662 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 01:49:16,732 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 01:49:16,762 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 01:49:16,762 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 01:49:16,762 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 01:49:16,832 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 01:49:16,862 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 01:49:16,862 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 01:49:16,862 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 01:49:16,932 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 01:49:16,962 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 01:49:16,962 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 01:49:16,962 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 01:49:17,32 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Creating NoOpTransaction because of error akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:49:17,33 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] NoOpTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:49:17,33 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] NoOpTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 directCommit called, failure org.opendaylight.controller.cluster.datastore.exceptions.ShardLeaderNotRespondingException: Could not create a READ_WRITE transaction on shard cars. The shard leader isn't responding. at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport.createTransactionContext(RemoteTransactionContextSupport.java:213) at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport.onCreateTransactionComplete(RemoteTransactionContextSupport.java:195) at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport$1.onComplete(RemoteTransactionContextSupport.java:136) at akka.dispatch.OnComplete.internal(Future.scala:299) at akka.dispatch.OnComplete.internal(Future.scala:297) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:49:17,33 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:49:17,34 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithShardLeaderNotResponding data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:49:17,34 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 01:49:17,34 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:49:17,34 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithShardLeaderNotResponding data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:49:17,35 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 01:49:17,35 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 01:49:17,35 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:49:17,35 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Closing 01:49:17,36 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Aborting 0 pending queued transactions 01:49:17,36 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 01:49:17,36 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:17,36 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:17,566 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [5308539937432240015] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:18,54 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:18,54 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:18,55 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:18.056] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:18,57 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4318444711393860697] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:18,63 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:18,63 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:18,65 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:18.066] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:18,96 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:18,103 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2405825671980353582] 01:49:18,104 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:18,105 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:18,105 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:18,106 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:18,106 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:18,114 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:18,115 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:18,115 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:18,117 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:18,129 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-3971492521017029570] 01:49:18,130 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:18,131 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:18,132 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:18,141 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:18,141 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:18,148 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:18,165 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:18,167 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:18,175 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [3550898492417192099] 01:49:18,175 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:18,181 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:18,181 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:18,182 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:18,182 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:18,188 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:18,189 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:18,191 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 01:49:18,196 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:49:18,200 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-722943324] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:18,201 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 01:49:18,201 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:18,201 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:18,201 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:18,206 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: no initial generation override, starting from 0 01:49:18,206 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:49:18,206 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1c1a209d to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6717c808 01:49:18,206 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@22154a8e 01:49:18,206 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1710164958206, None)) 01:49:18,207 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164958205,0,0)) 01:49:18,207 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5de4f54 01:49:18,207 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6717c808 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7c2ddbca 01:49:18,207 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} started 01:49:18,208 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:18,208 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:18,208 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:18,210 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:18,211 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 01:49:18,212 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:49:18,214 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:18,214 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:18,214 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: updated peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1043288434] 01:49:18,214 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 01:49:18,214 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1043288434] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:18,215 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-2079589976 created and ready for shard:member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:18,215 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-2079589976] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:18,215 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-175317522] 01:49:18,215 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-175317522] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:18,215 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 01:49:18,215 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:18,215 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:18,216 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 01:49:18,216 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 01:49:18,216 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:18,217 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1903293450] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:18,218 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:49:18,218 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:49:18,218 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 01:49:18,218 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 01:49:18,218 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 01:49:18,222 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:49:18,223 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: no initial generation override, starting from 0 01:49:18,223 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:49:18,223 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3db75c10 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@40c25cae 01:49:18,223 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5c0dad2 01:49:18,223 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1710164958223, None)) 01:49:18,223 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164958222,0,0)) 01:49:18,223 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 01:49:18,223 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@40c25cae to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@78d762c4 01:49:18,223 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:18,224 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} started 01:49:18,224 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:18,224 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:18,224 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@57dff66d 01:49:18,224 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:18,224 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:18,224 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:18,224 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:18,224 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: updated peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-712000771] 01:49:18,225 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 01:49:18,225 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-712000771] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:18,225 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-2:snapshot-read#1021595011] 01:49:18,225 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#864951066 created and ready for shard:member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:18,225 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#864951066] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:18,225 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-2:snapshot-read#1021595011] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:18,225 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 01:49:18,226 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:18,226 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:18,226 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 01:49:18,226 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 01:49:18,226 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:49:18,226 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:49:18,226 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 01:49:18,226 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:18,226 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 01:49:18,226 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 01:49:18,274 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-722943324] 01:49:18,274 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 01:49:18,274 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1043288434] 01:49:18,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:18,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 01:49:18,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:49:18,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:18,490 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 1 01:49:18,490 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:18,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:49:18,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:49:18,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:49:18,490 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 01:49:18,490 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 01:49:18,800 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:49:18,800 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:18,800 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 2 01:49:18,836 PM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:19,70 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:49:19,70 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,70 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 3 01:49:19,121 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:49:19,121 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:19,121 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:19,121 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,121 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,121 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:19,121 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,121 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,121 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,126 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:19,126 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,126 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,126 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:19,126 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,126 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,195 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:49:19,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:49:19,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,370 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 4 01:49:19,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,372 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:19,372 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:49:19,372 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:19,372 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:19,372 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=null 01:49:19,372 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:19,372 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,372 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:49:19,374 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:49:19,374 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:49:19,374 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:49:19,374 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:19,374 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:49:19,374 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 01:49:19,374 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [] 01:49:19,374 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Leader 01:49:19,374 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 01:49:19,374 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:19,374 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:19,374 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Leader 01:49:19,374 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 01:49:19,375 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 01:49:19,375 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,375 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 01:49:19,375 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:19,375 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:19,375 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 01:49:19,375 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 01:49:19,375 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: -1, nextIndex: 0 01:49:19,376 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:19,382 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testSingleShardTransactionsWithLeaderChanges commit rateLimit adjusted to 95.19145652890465 01:49:19,382 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:49:19,382 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1903293450] 01:49:19,382 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Queuing TransactionOperation 01:49:19,382 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Queuing TransactionOperation 01:49:19,382 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Readying 1 components for commit 01:49:19,382 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Readying transaction for shard cars 01:49:19,383 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Queuing TransactionOperation 01:49:19,383 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 canCommit 01:49:19,383 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:19,383 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 01:49:19,383 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 01:49:19,383 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges)] for shard cars 01:49:19,383 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges)] found - creating WRITE_ONLY transaction context 01:49:19,383 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:49:19,383 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:49:19,383 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 directCommit called 01:49:19,383 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 sending 2 batched modifications, ready: true 01:49:19,383 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges)] 01:49:19,387 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, disabling transaction tracking 01:49:19,387 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 01:49:19,387 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 01:49:19,387 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:49:19,387 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:19,387 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,387 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}]]] 01:49:19,387 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying 2 batched modifications for Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Readying Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 of 2 operations, client version 13 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: readying transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 ready 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Validating transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 validated 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: canCommit for member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: success 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Preparing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 candidate ready 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: canCommit - messages: [] 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 prepared 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Finishing commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Starting commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 181 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 01:49:19,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:19,389 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:19,389 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,389 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}]]] 01:49:19,389 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 submitted to persistence 01:49:19,389 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 01:49:19,389 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:49:19,389 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71} 01:49:19,389 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 01:49:19,389 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 1 01:49:19,389 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:19,389 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 01:49:19,389 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 01:49:19,389 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 01:49:19,389 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 01:49:19,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 0, nextIndex: 1 01:49:19,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:49:19,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 0 01:49:19,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 01:49:19,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71} 01:49:19,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:49:19,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} does not match any client, pre-disabling client. 01:49:19,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, nanosAgo=2955219} is already disabled 01:49:19,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:19,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}]]] 01:49:19,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=0 01:49:19,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}]]] 01:49:19,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 01:49:19,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:49:19,392 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log Simple{size=181} 01:49:19,392 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 01:49:19,392 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 2 01:49:19,392 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 0 01:49:19,392 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:19,392 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:49:19,392 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 0 01:49:19,392 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 01:49:19,392 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71} 01:49:19,392 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:49:19,392 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} does not match any client, pre-disabling client. 01:49:19,392 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 01:49:19,392 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=0 01:49:19,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 1, nextIndex: 2 01:49:19,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:49:19,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 1 01:49:19,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 01:49:19,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 1 data Simple{size=181} 01:49:19,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Resuming commit of transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:49:19,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:49:19,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:49:19,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleShardTransactionsWithLeaderChanges_member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges$a] 01:49:19,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#241525097] with name member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 01:49:19,393 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#241525097] 01:49:19,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 0 for term 4 01:49:19,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:19,393 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 01:49:19,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=1 01:49:19,393 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#241525097] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:19,393 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 99.92 μs 01:49:19,394 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 modifications completed with CommitTransactionReply [version=13] 01:49:19,394 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 successfully completed direct commit 01:49:19,394 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 1 01:49:19,394 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:49:19,394 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 1 01:49:19,394 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 01:49:19,394 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 1 data Simple{size=181} 01:49:19,394 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying foreign transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:49:19,394 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-1003642336] with name member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 01:49:19,395 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 0 for term 4 01:49:19,395 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=1 01:49:19,395 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testSingleShardTransactionsWithLeaderChanges 01:49:19,395 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-1003642336] 01:49:19,395 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 01:49:19,395 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-1003642336] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:19,395 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 78.21 μs 01:49:19,395 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,395 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 01:49:19,395 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Re-initializing to Follower with previous leaderId member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,395 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:49:19,395 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : true 01:49:19,395 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:49:19,395 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:49:19,396 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,396 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-545: Closing 01:49:19,396 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 01:49:19,396 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:49:19,396 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:19,397 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:19,401 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:19,401 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:19,401 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:19,401 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:19,403 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:19,411 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 01:49:19,461 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1903293450] 01:49:19,461 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 01:49:19,461 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-712000771] 01:49:19,461 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,511 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,561 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,586 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 01:49:19,586 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 01:49:19,586 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 01:49:19,586 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 01:49:19,586 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:49:19,586 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,586 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 5 01:49:19,586 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,586 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:49:19,586 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:49:19,587 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:49:19,587 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:49:19,587 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 01:49:19,587 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 01:49:19,587 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=-1] 01:49:19,587 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:19,587 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:19,587 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 01:49:19,618 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:19,626 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2215113898063567499] 01:49:19,626 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:19,626 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:19,626 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:19,632 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:19,632 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:19,633 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:19,634 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:19,636 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 01:49:19,640 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:49:19,641 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#572970556] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:19,642 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: recovered identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:49:19,643 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 01:49:19,643 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,643 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} 01:49:19,643 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@66046e6c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3402ba96 01:49:19,643 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@71003272 01:49:19,643 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1710164959643, None)) 01:49:19,643 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164959642,0,0)) 01:49:19,643 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6f004133 01:49:19,643 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3402ba96 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@61cce0d4 01:49:19,644 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} started 01:49:19,644 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:19,644 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,644 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 01:49:19,644 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#2119851561] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:19,645 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 01:49:19,655 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#197196058] 01:49:19,655 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#28458805 created and ready for shard:member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,655 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#28458805] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:19,655 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#197196058] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:19,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:49:19,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:49:19,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:49:19,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:49:19,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 01:49:19,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received ReplicatedLogEntry for recovery: index: 0, size: 71 01:49:19,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 01:49:19,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received ReplicatedLogEntry for recovery: index: 1, size: 181 01:49:19,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 01:49:19,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received apply journal entries for recovery, applying to state: 0 to 0 01:49:19,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: starting log recovery batch with max size 1 01:49:19,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: ignoring unhandled payload DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71} 01:49:19,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 01:49:19,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 01:49:19,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received apply journal entries for recovery, applying to state: 1 to 1 01:49:19,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: starting log recovery batch with max size 1 01:49:19,658 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@442d4d13}, purged=MutableUnsignedLongSet{size=0}} 01:49:19,658 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:49:19,658 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 01:49:19,658 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:19,658 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed in 707.2 μs - Switching actor to Follower - last log index = 1, last log term = 4, snapshot index = -1, snapshot term = -1, journal size = 2 01:49:19,658 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:49:19,658 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:49:19,658 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 01:49:19,658 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 01:49:19,658 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 01:49:19,658 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 01:49:19,658 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 01:49:19,694 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1903293450] 01:49:19,694 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 01:49:19,694 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-712000771] 01:49:19,746 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:49:19,746 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,746 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 6 01:49:19,746 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,915 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:49:19,915 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:19,915 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 7 01:49:19,916 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:20,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:49:20,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:20,75 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 8 01:49:20,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:20,206 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(akka://cluster-test@127.0.0.1:2558, Removed) 01:49:20,206 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 01:49:20,206 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:49:20,206 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:20,206 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 5 01:49:20,207 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:49:20,207 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:49:20,207 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:49:20,207 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:49:20,207 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 01:49:20,207 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 01:49:20,225 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:49:20,225 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:20,226 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 9 01:49:20,226 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:20,416 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:49:20,416 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:20,416 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 10 01:49:20,416 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:20,417 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2405825671980353582] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:20,418 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [2405825671980353582] but current is [2215113898063567499]. Reason: ActorSystem terminated 01:49:20,452 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] LocalActorRefProvider$$anon$2 - Message [akka.cluster.GossipEnvelope] wrapped in [akka.actor.ActorSelectionMessage] to Actor[akka://cluster-test/] was dropped. Unknown system with UID [3550898492417192099]. This system with UID [2215113898063567499] was probably restarted. Messages will be accepted when new handshake has been completed.. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:20,462 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2558, Leaving)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 01:49:20,463 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking unreachable node [akka://cluster-test@127.0.0.1:2558] as [Down] 01:49:20,465 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:20,502 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#1229842874]] intended for system uid [2405825671980353582], self uid is [2215113898063567499] 01:49:20,502 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4], my term is 5 01:49:20,502 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastIndex: 1, lastTerm: 4 01:49:20,502 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=5, voteGranted=false] 01:49:20,502 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4], my term is 5 01:49:20,502 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 6 in "RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than Candidate's term 5 - switching to Follower 01:49:20,502 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=null 01:49:20,502 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 1, lastTerm: 4 01:49:20,502 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:20,502 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 01:49:20,502 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 6 01:49:20,502 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:49:20,502 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:49:20,503 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 01:49:20,503 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 01:49:20,504 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:20,504 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:49:20,504 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 01:49:20,504 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 7 in "RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 6 - updating term 01:49:20,504 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=null 01:49:20,505 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 7, votedFor: null, lastIndex: 1, lastTerm: 4 01:49:20,505 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:20,505 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=7, voteGranted=true] 01:49:20,505 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=false], my term is 10 01:49:20,505 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=false], current voteCount: 1 01:49:20,505 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:20,505 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=true], my term is 10 01:49:20,505 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=true], current voteCount: 1 01:49:20,505 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:49:20,505 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 01:49:20,505 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 8 in "RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 7 - updating term 01:49:20,505 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=null 01:49:20,505 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:49:20,505 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 8, votedFor: null, lastIndex: 1, lastTerm: 4 01:49:20,505 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:20,505 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=8, voteGranted=true] 01:49:20,505 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:20,505 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Leader, election term: 10 01:49:20,505 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 01:49:20,505 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges] 01:49:20,505 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Leader 01:49:20,505 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 01:49:20,506 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:20,506 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:20,506 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Leader 01:49:20,506 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 01:49:20,506 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:20,506 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:49:20,506 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 01:49:20,506 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 9 in "RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 8 - updating term 01:49:20,506 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=null 01:49:20,506 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 9, votedFor: null, lastIndex: 1, lastTerm: 4 01:49:20,506 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:20,506 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=9, voteGranted=true] 01:49:20,507 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:20,507 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:49:20,507 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 01:49:20,507 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 10 in "RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 9 - updating term 01:49:20,507 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=null 01:49:20,507 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 10, votedFor: null, lastIndex: 1, lastTerm: 4 01:49:20,507 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:20,507 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=10, voteGranted=true] 01:49:20,507 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 01:49:20,507 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = null 01:49:20,507 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 01:49:20,507 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 01:49:20,507 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:20,507 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:20,507 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 01:49:20,508 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 1, nextIndex: 2 01:49:20,508 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:20,509 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - New leader address: akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:20,509 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:20,509 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 0 for term 10 01:49:20,509 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 01:49:20,550 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:49:20,550 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1903293450] 01:49:20,551 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 Queuing TransactionOperation 01:49:20,551 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 Readying 1 components for commit 01:49:20,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:20,551 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 Readying transaction for shard cars 01:49:20,551 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 Queuing TransactionOperation 01:49:20,551 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 canCommit 01:49:20,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-712000771, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@303f53c9] 01:49:20,551 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-712000771, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@303f53c9] 01:49:20,551 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-712000771)] for shard cars 01:49:20,551 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 01:49:20,551 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@35dc9334 01:49:20,552 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 = [ 1 ] 01:49:20,552 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-712000771)] 01:49:20,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: handleReadyLocalTransaction for member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:49:20,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying local modifications for Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:49:20,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Validating transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:49:20,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 validated 01:49:20,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:20,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: canCommit for member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: success 01:49:20,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:49:20,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Preparing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:49:20,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 candidate ready 01:49:20,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: canCommit - candidate rootPath: / 01:49:20,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: canCommit - messages: [] 01:49:20,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 01:49:20,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 prepared 01:49:20,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:20,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Finishing commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:49:20,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Starting commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:49:20,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 01:49:20,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}] 01:49:20,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}] 01:49:20,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:20,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:49:20,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 2 is present for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:20,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}]]] 01:49:20,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 submitted to persistence 01:49:20,557 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}]]] 01:49:20,557 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 01:49:20,557 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:49:20,557 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log Simple{size=240} 01:49:20,557 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}] 01:49:20,557 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 2 01:49:20,557 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=10, success=true, followerId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=2, logLastTerm=10, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:20,561 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}] 01:49:20,561 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}] 01:49:20,562 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 2, nextIndex: 3 01:49:20,562 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:49:20,562 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 2 01:49:20,562 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 1 to 2 01:49:20,562 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 2 data Simple{size=240} 01:49:20,562 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Resuming commit of transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:49:20,562 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: commit - successfulFromPrevious: [] 01:49:20,562 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 01:49:20,562 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSingleShardTransactionsWithLeaderChanges_member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges%23-712000771$b] 01:49:20,562 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 successfully completed direct commit 01:49:20,562 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 committed successfully 01:49:20,562 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 01:49:20,562 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0 01:49:20,562 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 1 for term 10 01:49:20,562 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-712000771)] for shard cars, trying to use DirectTransactionContextWrapper 01:49:20,562 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@35dc9334 01:49:20,562 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0: Local transaction context created successfully, using DirectTransactionWrapper 01:49:20,562 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:49:20,563 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 141.6 μs 01:49:20,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=2 01:49:20,563 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} 01:49:20,563 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Issuing gracefulStop to shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:20,563 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges - waiting on 1 shards 01:49:20,563 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onShutDown 01:49:20,563 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:20,563 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:49:20,563 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 01:49:20,564 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All shards shutdown - sending PoisonPill to self 01:49:20,564 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:49:20,564 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:49:20,564 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:49:20,564 PM [main] [WARN] ActorUtils - ShardManager for testSingleShardTransactionsWithLeaderChanges data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:49:20,564 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:49:20,565 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:49:20,565 PM [main] [WARN] ActorUtils - ShardManager for testSingleShardTransactionsWithLeaderChanges data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:49:20,565 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-712000771] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:20,565 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: shutting down 01:49:20,565 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:49:20,565 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:49:20,565 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#572970556] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#572970556] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:20,565 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#572970556] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#572970556] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:20,565 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#-591911874] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-591911874] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:20,568 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:20,568 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:20,572 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:20,572 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:20,572 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:20,572 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:20,573 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:20,588 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:49:20,588 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:49:20,588 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-548: Closing 01:49:20,588 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 01:49:20,589 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:49:20,590 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:20,590 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:21,237 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is removing unreachable node [akka://cluster-test@127.0.0.1:2558] 01:49:21,239 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [2405825671980353582] but current is [2215113898063567499]. Reason: Cluster member removed, previous status [Down] 01:49:21,565 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [2405825671980353582] but current is [2215113898063567499]. Reason: Cluster member removed, previous status [Down] 01:49:21,585 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2215113898063567499] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:21,605 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-3971492521017029570] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:21,607 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:21,608 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:21,609 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:21.610] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:21,617 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:21,617 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:21,619 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:21.620] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:21,635 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:21,640 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-2415868336465767284] 01:49:21,640 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:21,642 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:21,642 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:21,648 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:21,648 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:21,652 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:21,652 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:21,652 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:21,656 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:21,661 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-7332831146606244450] 01:49:21,662 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:21,663 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:21,663 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:21,664 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:21,664 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:21,678 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:21,685 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [5474887455762468373] 01:49:21,685 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:21,689 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:21,689 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:21,689 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:21,689 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:21,697 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:21,698 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:21,699 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 01:49:21,711 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-257432825] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:21,713 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: no initial generation override, starting from 0 01:49:21,713 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:49:21,713 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4cbbd158 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7ec7e8d7 01:49:21,713 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@16f9e502 01:49:21,713 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1710164961713, None)) 01:49:21,713 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164961712,0,0)) 01:49:21,713 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7ec7e8d7 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7651714d 01:49:21,714 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} started 01:49:21,714 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:21,716 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:21,718 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 01:49:21,719 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 01:49:21,721 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 01:49:21,721 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:21,721 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:21,721 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:21,721 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@28f1bca5 01:49:21,721 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:21,721 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:21,721 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:21,721 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:21,721 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-1-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904] 01:49:21,722 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithIsolatedLeader, persistent : true 01:49:21,722 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 01:49:21,723 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:21,723 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/shard-member-1:snapshot-read#-133680235] 01:49:21,723 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#1817728019 created and ready for shard:member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:21,723 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#1817728019] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:21,723 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/shard-member-1:snapshot-read#-133680235] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:21,724 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:21,724 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:21,724 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 01:49:21,724 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithIsolatedLeader is initialized 01:49:21,724 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:49:21,724 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 01:49:21,724 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 01:49:21,724 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 01:49:21,724 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from null to Follower 01:49:21,728 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2025870202] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:21,730 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 01:49:21,733 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: no initial generation override, starting from 0 01:49:21,733 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:49:21,733 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 01:49:21,733 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4a163182 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@22e310d 01:49:21,733 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@611a590 01:49:21,733 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:21,733 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1710164961733, None)) 01:49:21,733 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164961732,0,0)) 01:49:21,733 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@44937929 01:49:21,733 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@22e310d to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@729a9c17 01:49:21,734 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} started 01:49:21,734 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:21,734 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:21,735 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithIsolatedLeader, persistent : true 01:49:21,735 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 01:49:21,735 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#1079760] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:21,735 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#628728050] 01:49:21,735 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#785127557 created and ready for shard:member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:21,735 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#628728050] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:21,737 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:21,737 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:21,737 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 01:49:21,737 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithIsolatedLeader is initialized 01:49:21,737 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:49:21,737 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 01:49:21,737 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 01:49:21,737 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 01:49:21,737 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-2-shard-cars-testTransactionWithIsolatedLeader from null to Follower 01:49:21,737 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#785127557] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:21,750 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:21,751 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:21,752 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:21,752 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:21,752 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:21,752 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:21,755 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:49:21,756 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:21,784 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-257432825] 01:49:21,784 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: findLocalShard : cars 01:49:21,784 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904] 01:49:22,44 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:22,44 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Received ElectionTimeout - switching to Candidate 01:49:22,44 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 01:49:22,44 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:22,44 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 1 01:49:22,44 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:22,44 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 01:49:22,44 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:49:22,44 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 01:49:22,45 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Follower to Candidate 01:49:22,45 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Follower to Candidate 01:49:22,304 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:22,343 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 01:49:22,343 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:22,343 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 2 01:49:22,563 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 01:49:22,563 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:22,563 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 3 01:49:22,664 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:49:22,664 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:22,665 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:22,665 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 01:49:22,665 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:22,665 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:22,665 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:22,665 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:22,665 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:22,678 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:22,678 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 01:49:22,678 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:22,678 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:22,678 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:22,678 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:22,704 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:49:22,823 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 01:49:22,823 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:22,823 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 4 01:49:22,823 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:22,824 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:22,825 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Cluster state: [] 01:49:22,825 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:22,825 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:22,825 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=null 01:49:22,825 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:22,825 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:22,825 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:49:22,826 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:49:22,826 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:49:22,826 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 01:49:22,826 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:22,826 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:49:22,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:49:22,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: became leader with frontend state for [] 01:49:22,827 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Candidate to Leader 01:49:22,827 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 01:49:22,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:22,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:22,827 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Candidate to Leader 01:49:22,827 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 01:49:22,827 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader: Closing 01:49:22,827 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithIsolatedLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:22,828 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done false 01:49:22,828 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 01:49:22,828 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:22,828 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:22,828 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 01:49:22,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: -1, nextIndex: 0 01:49:22,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:22,846 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionWithIsolatedLeader commit rateLimit adjusted to 95.19145652890465 01:49:22,846 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:49:22,846 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-257432825] 01:49:22,846 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:22,846 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 Queuing TransactionOperation 01:49:22,846 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@63e526ac] 01:49:22,846 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 01:49:22,846 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-257432825] 01:49:22,846 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@63e526ac] 01:49:22,846 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 Queuing TransactionOperation 01:49:22,846 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:49:22,846 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-257432825] 01:49:22,846 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904)] for shard cars 01:49:22,846 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 Queuing TransactionOperation 01:49:22,846 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 01:49:22,846 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@100c2b27 01:49:22,847 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:22,847 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@63e526ac] 01:49:22,847 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:22,847 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@63e526ac] 01:49:22,847 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@63e526ac] 01:49:22,847 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@63e526ac] 01:49:22,847 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904)] for shard cars 01:49:22,847 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@100c2b27 01:49:22,847 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904)] for shard cars 01:49:22,847 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@100c2b27 01:49:22,857 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:49:22,857 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:22,857 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 01:49:22,857 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onShutDown 01:49:22,857 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:22,857 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 01:49:22,858 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 01:49:22,858 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 01:49:22,858 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 01:49:22,858 PM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:49:22,858 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 Readying 1 components for commit 01:49:22,858 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 Readying transaction for shard cars 01:49:22,858 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 = [ 1 ] 01:49:22,858 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904)] 01:49:22,858 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: handleReadyLocalTransaction for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, disabling transaction tracking 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}]]] 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying local modifications for Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 validated 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: canCommit for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: success 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Committing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Preparing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 candidate ready 01:49:22,859 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithIsolatedLeader 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: canCommit - messages: [] 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 prepared 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Finishing commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:49:22,859 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Starting commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:49:22,860 PM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 170 01:49:22,860 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 01:49:22,860 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 01:49:22,860 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:22,860 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:22,860 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:22,860 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 submitted to persistence 01:49:22,860 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: shutting down 01:49:22,860 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:49:22,860 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2025870202] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2025870202] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:22,860 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#-1271165341] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1271165341] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:22,860 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:22,861 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-257432825] 01:49:22,861 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: findLocalShard : cars 01:49:22,861 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904] 01:49:22,861 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 01:49:22,861 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904] 01:49:22,861 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 01:49:22,861 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 01:49:22,862 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:22,862 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2025870202] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2025870202] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:22,912 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904] 01:49:22,943 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:22,962 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904] 01:49:23,13 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904] 01:49:23,63 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904] 01:49:23,64 PM [cluster-test-shard-dispatcher-11] [WARN] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: At least 1 followers need to be active, Switching member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 01:49:23,65 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:23,66 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 01:49:23,66 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:23,66 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 4 01:49:23,66 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader)_slicer-id-553: Closing 01:49:23,66 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:49:23,66 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Leader to IsolatedLeader 01:49:23,67 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:23,69 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 01:49:23,114 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904] 01:49:23,114 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 Readying 1 components for commit 01:49:23,114 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 Readying transaction for shard cars 01:49:23,114 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 = [ 1 ] 01:49:23,114 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904)] 01:49:23,115 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: handleReadyLocalTransaction for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 01:49:23,115 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionWithIsolatedLeader: Adding message org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction@65d693d6 to retry 01:49:23,115 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 canCommit 01:49:23,533 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionWithIsolatedLeader: Timer expired for message org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction@65d693d6 01:49:23,533 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] LocalThreePhaseCommitCohort - Failed to prepare transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 on backend org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Could not process ready local transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0. Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.ShardTransactionMessageRetrySupport$MessageInfo.timedOut(ShardTransactionMessageRetrySupport.java:102) at org.opendaylight.controller.cluster.datastore.ShardTransactionMessageRetrySupport.onTimerMessage(ShardTransactionMessageRetrySupport.java:72) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:410) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:49:23,534 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithIsolatedLeader 01:49:23,534 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 Readying 1 components for commit 01:49:23,534 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 Readying transaction for shard cars 01:49:23,534 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 = [ 1 ] 01:49:23,534 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1341470904)] 01:49:23,535 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:23,536 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: handleReadyLocalTransaction for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:49:23,536 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionWithIsolatedLeader: Adding message org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction@50fa0932 to retry 01:49:23,536 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:23,536 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 01:49:23,537 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:23,539 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 01:49:23,540 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 01:49:23,541 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: recovered identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:49:23,541 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} 01:49:23,541 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 01:49:23,541 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:23,541 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@47ef1870 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@71e43384 01:49:23,541 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4c1814e8 01:49:23,541 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:23,541 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 01:49:23,541 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1710164963541, None)) 01:49:23,541 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:23,541 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:23,541 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:23,541 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:23,542 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164963540,0,0)) 01:49:23,542 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@553fc56 01:49:23,542 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@71e43384 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@46c3381c 01:49:23,542 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} started 01:49:23,542 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 canCommit 01:49:23,542 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:23,542 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:23,543 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:23,543 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:23,543 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#1707771324] 01:49:23,543 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:23,543 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:23,543 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-2-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#1707771324] 01:49:23,543 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 01:49:23,543 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:23,543 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-3-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#1707771324] 01:49:23,543 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithIsolatedLeader, persistent : true 01:49:23,543 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#-533434922] 01:49:23,543 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#-887598801 created and ready for shard:member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:23,543 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 01:49:23,544 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:23,544 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}]]] 01:49:23,544 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:49:23,544 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=null 01:49:23,544 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:49:23,544 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:23,544 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:23,544 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:23,544 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:49:23,544 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 01:49:23,545 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:23,545 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 01:49:23,545 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithIsolatedLeader is initialized 01:49:23,545 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 01:49:23,545 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 01:49:23,545 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-2-shard-cars-testTransactionWithIsolatedLeader from null to Follower 01:49:23,546 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}]]] 01:49:23,546 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 2 01:49:23,546 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:49:23,546 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60} 01:49:23,546 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 01:49:23,546 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log Simple{size=170} 01:49:23,546 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 01:49:23,546 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 2 01:49:23,546 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:23,546 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader: Closing 01:49:23,546 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithIsolatedLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:23,546 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 01:49:23,546 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:23,546 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:23,546 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 01:49:23,547 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 01:49:23,547 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 01:49:23,547 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done false 01:49:23,547 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 1, nextIndex: 2 01:49:23,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 01:49:23,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Setting last applied to 0 01:49:23,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from -1 to 0 01:49:23,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60} 01:49:23,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:49:23,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionWithIsolatedLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} does not match any client, pre-disabling client. 01:49:23,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, nanosAgo=689112785} is already disabled 01:49:23,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Setting last applied to 1 01:49:23,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 0 to 1 01:49:23,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 1 data Simple{size=170} 01:49:23,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Resuming commit of transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:49:23,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:49:23,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:49:23,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionWithIsolatedLeader_member-1-shard-cars-testTransactionWithIsolatedLeader%23-1341470904$a] 01:49:23,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionWithIsolatedLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-790223139] with name member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher 01:49:23,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 0 for term 4 01:49:23,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:23,548 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - IsolatedLeader member-1-shard-cars-testTransactionWithIsolatedLeader switching from IsolatedLeader to Leader 01:49:23,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 4 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader)_slicer-id-554: Closing 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionWithIsolatedLeader: Retrying message org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction@50fa0932 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=1 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: handleReadyLocalTransaction for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:49:23,549 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 committed successfully 01:49:23,549 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-790223139] 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying local modifications for Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:49:23,549 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 successfully completed direct commit 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 validated 01:49:23,549 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from IsolatedLeader to Leader 01:49:23,549 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: canCommit for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: success 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Committing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Preparing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:49:23,549 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-790223139] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 candidate ready 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: canCommit - candidate rootPath: / 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 01:49:23,549 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 184.5 μs 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: canCommit - messages: [] 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 prepared 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Finishing commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Starting commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 68 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 01:49:23,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 01:49:23,550 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 canCommit 01:49:23,550 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:23,550 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:49:23,550 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:23,550 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}]]] 01:49:23,550 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from IsolatedLeader to Leader 01:49:23,550 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 submitted to persistence 01:49:23,550 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 01:49:23,551 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 01:49:23,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 1 01:49:23,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 01:49:23,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 0 01:49:23,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from -1 to 0 01:49:23,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60} 01:49:23,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:49:23,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionWithIsolatedLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} does not match any client, pre-disabling client. 01:49:23,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 1 01:49:23,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 0 to 1 01:49:23,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 1 data Simple{size=170} 01:49:23,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying foreign transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:49:23,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionWithIsolatedLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-1345192479] with name member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher 01:49:23,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 0 for term 4 01:49:23,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}]]] 01:49:23,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 1 01:49:23,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:49:23,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log Simple{size=68} 01:49:23,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 01:49:23,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 2 01:49:23,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:23,552 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-1345192479] 01:49:23,552 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 01:49:23,552 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 52.06 μs 01:49:23,552 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 01:49:23,552 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=1 01:49:23,552 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done true 01:49:23,552 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:23,552 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:23,552 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 2, nextIndex: 3 01:49:23,552 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:49:23,552 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 2 01:49:23,552 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 1 to 2 01:49:23,552 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 2 data Simple{size=68} 01:49:23,552 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Resuming commit of transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:49:23,553 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: commit - successfulFromPrevious: [] 01:49:23,553 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 01:49:23,553 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionWithIsolatedLeader_member-1-shard-cars-testTransactionWithIsolatedLeader%23-1341470904$b] 01:49:23,553 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 1 for term 4 01:49:23,553 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:49:23,553 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=2 01:49:23,553 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 successfully completed direct commit 01:49:23,553 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 committed successfully 01:49:23,553 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 01:49:23,553 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 15.40 μs 01:49:23,553 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:49:23,553 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 2 01:49:23,553 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:49:23,553 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 2 01:49:23,553 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 1 to 2 01:49:23,553 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 2 data Simple{size=68} 01:49:23,553 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying foreign transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:49:23,553 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 1 for term 4 01:49:23,553 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:23,553 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=2 01:49:23,553 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 01:49:23,553 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 01:49:23,553 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 11.80 μs 01:49:23,553 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onShutDown 01:49:23,553 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Initiating leader transfer 01:49:23,554 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6fbd7e01 01:49:23,554 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6fbd7e01 01:49:23,554 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Attempting to transfer leadership 01:49:23,554 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:49:23,554 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderId=null, leaderPayloadVersion=13] 01:49:23,554 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:23,554 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:23,554 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionWithIsolatedLeader, matchIndex: 2, lastIndex: 2, isVoting: true 01:49:23,554 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Follower's log matches - sending ElectionTimeout 01:49:23,554 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:49:23,554 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Leader transfer complete 01:49:23,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader] 01:49:23,555 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=null, leaderPayloadVersion=13] 01:49:23,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:23,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:23,555 PM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:23,684 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:49:25,574 PM [cluster-test-shard-dispatcher-13] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testTransactionWithIsolatedLeader: Successfully transferred leadership to member-1-shard-cars-testTransactionWithIsolatedLeader in 2.020 s 01:49:25,574 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: leader transfer succeeded - sending PoisonPill 01:49:25,574 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithIsolatedLeader 01:49:25,574 PM [cluster-test-shard-dispatcher-13] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader)_slicer-id-557: Closing 01:49:25,574 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 01:49:25,574 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 01:49:25,575 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 01:49:25,575 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:49:25,575 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:49:25,575 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: shutting down 01:49:25,575 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:49:25,575 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:49:25,575 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-257432825] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-257432825] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:25,575 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-257432825] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-257432825] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:25,575 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-257432825] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-257432825] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:25,575 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#-1824895422] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1824895422] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:25,576 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:25,576 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#-1824895422] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1824895422] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:25,576 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:25,582 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:25,582 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:25,582 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:25,583 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:25,584 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:25,595 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:49:25,595 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithIsolatedLeader 01:49:25,595 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 01:49:25,595 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 01:49:25,595 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 01:49:25,596 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:25,596 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:26,595 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-2415868336465767284] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:26,615 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-7332831146606244450] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:26,616 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:26,616 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:26,618 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:26.619] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:26,634 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:26,634 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:26,639 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:26.641] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:26,660 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:26,669 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-1071623744681294309] 01:49:26,669 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:26,672 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:26,672 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:26,672 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:26,672 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:26,681 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:26,681 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:26,681 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:26,690 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:26,699 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [5696047322523677005] 01:49:26,699 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:26,701 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:26,701 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:26,701 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:26,706 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:26,740 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:26,742 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:26,746 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:26,752 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [421980979009161151] 01:49:26,752 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:26,756 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:26,756 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:26,756 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:26,756 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:26,775 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 closeTransaction called 01:49:26,775 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_30#-1285259706)] 01:49:26,780 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:26,782 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:26,784 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithMultipleShards 01:49:26,792 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-1218102381] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:26,797 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:49:26,799 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: no initial generation override, starting from 0 01:49:26,799 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:49:26,799 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7830f91f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@91b8206 01:49:26,799 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:26,799 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4c190d41 01:49:26,800 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testWriteTransactionWithMultipleShards, 0, 1710164966799, None)) 01:49:26,800 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164966798,0,0)) 01:49:26,800 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@91b8206 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1c86e8d7 01:49:26,800 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} started 01:49:26,801 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:26,803 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:26,804 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithMultipleShards 01:49:26,806 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithMultipleShards 01:49:26,808 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-1915136545] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:26,808 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithMultipleShards 01:49:26,808 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-cars-testWriteTransactionWithMultipleShards 01:49:26,808 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-people-testWriteTransactionWithMultipleShards 01:49:26,808 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6b4182 01:49:26,808 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:26,808 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testWriteTransactionWithMultipleShards 01:49:26,809 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithMultipleShards 01:49:26,809 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testWriteTransactionWithMultipleShards 01:49:26,809 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:26,809 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:49:26,809 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:49:26,809 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:49:26,809 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:49:26,809 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testWriteTransactionWithMultipleShards, persistent : true 01:49:26,809 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-191211768] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:26,810 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-notifier#-1144862244 created and ready for shard:member-1-shard-cars-testWriteTransactionWithMultipleShards 01:49:26,810 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-notifier#-1144862244] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:26,810 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#1715540624] 01:49:26,810 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#1715540624] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:26,810 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testWriteTransactionWithMultipleShards with journal batch size 1 01:49:26,810 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-people-testWriteTransactionWithMultipleShards, persistent : true 01:49:26,811 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1996875763] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:26,811 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#272554275] 01:49:26,811 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-notifier#-1763451647 created and ready for shard:member-1-shard-people-testWriteTransactionWithMultipleShards 01:49:26,811 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-people-testWriteTransactionWithMultipleShards with journal batch size 1 01:49:26,811 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-notifier#-1763451647] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:26,811 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#272554275] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:26,811 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: no initial generation override, starting from 0 01:49:26,811 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:49:26,811 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2faa802c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@29bb694f 01:49:26,811 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2d84f8a0 01:49:26,811 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithMultipleShards, 0, 1710164966811, None)) 01:49:26,811 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithMultipleShards 01:49:26,811 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-cars-testWriteTransactionWithMultipleShards 01:49:26,812 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-people-testWriteTransactionWithMultipleShards 01:49:26,812 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:26,812 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:49:26,812 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:49:26,812 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164966810,0,0)) 01:49:26,812 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5dc2259 01:49:26,812 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@29bb694f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@53efd8f1 01:49:26,812 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} started 01:49:26,812 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:26,812 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithMultipleShards 01:49:26,812 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:49:26,812 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:49:26,812 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards on shard member-2-shard-cars-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards#156366319] 01:49:26,812 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testWriteTransactionWithMultipleShards 01:49:26,813 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards#156366319] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:26,813 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithMultipleShards, persistent : true 01:49:26,813 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-notifier#-772898897 created and ready for shard:member-2-shard-cars-testWriteTransactionWithMultipleShards 01:49:26,813 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-14624286] 01:49:26,813 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-notifier#-772898897] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:26,813 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-14624286] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:26,813 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithMultipleShards with journal batch size 1 01:49:26,813 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-people-testWriteTransactionWithMultipleShards, persistent : true 01:49:26,813 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards#1572861013] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:26,813 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-notifier#-1738719644 created and ready for shard:member-2-shard-people-testWriteTransactionWithMultipleShards 01:49:26,813 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#1870370407] 01:49:26,813 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-notifier#-1738719644] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:26,813 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#1870370407] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:26,813 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-people-testWriteTransactionWithMultipleShards with journal batch size 1 01:49:26,814 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:49:26,814 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:49:26,814 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards on shard member-2-shard-people-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards#1572861013] 01:49:26,815 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:26,815 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:26,815 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:26,815 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 01:49:26,815 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:26,815 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:26,815 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [cars] 01:49:26,815 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-1-shard-cars-testWriteTransactionWithMultipleShards is initialized 01:49:26,815 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:26,815 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:26,815 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:26,815 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 01:49:26,815 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:26,815 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:26,815 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 01:49:26,815 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:26,815 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-1-shard-people-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:49:26,815 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:49:26,815 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [people] 01:49:26,815 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-people-testWriteTransactionWithMultipleShards is initialized 01:49:26,815 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:26,815 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithMultipleShards , received role change from null to Follower 01:49:26,815 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 01:49:26,815 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [cars] 01:49:26,815 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithMultipleShards is initialized 01:49:26,815 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithMultipleShards , received role change from null to Follower 01:49:26,815 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from null to Follower 01:49:26,815 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from null to Follower 01:49:26,815 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 01:49:26,815 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 01:49:26,815 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 01:49:26,815 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-2-shard-cars-testWriteTransactionWithMultipleShards from null to Follower 01:49:26,815 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-2-shard-people-testWriteTransactionWithMultipleShards from null to Follower 01:49:26,818 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [people] 01:49:26,818 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-1-shard-people-testWriteTransactionWithMultipleShards is initialized 01:49:26,818 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from null to Follower 01:49:26,818 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 01:49:26,818 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from null to Follower 01:49:26,862 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-1218102381] 01:49:26,862 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: findLocalShard : cars 01:49:26,863 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-191211768] 01:49:27,89 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:27,89 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:49:27,89 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testWriteTransactionWithMultipleShards] 01:49:27,89 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 01:49:27,89 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 01:49:27,89 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:27,89 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Closing 01:49:27,89 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:27,89 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 01:49:27,89 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from Follower to Candidate 01:49:27,89 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from Follower to Candidate 01:49:27,118 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:27,118 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:49:27,118 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testWriteTransactionWithMultipleShards] 01:49:27,118 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 01:49:27,118 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 01:49:27,118 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:27,118 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Closing 01:49:27,118 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:27,118 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 01:49:27,118 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from Follower to Candidate 01:49:27,118 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from Follower to Candidate 01:49:27,353 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:27,368 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:49:27,368 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 01:49:27,368 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 01:49:27,382 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:27,388 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:49:27,389 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 01:49:27,389 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 01:49:27,658 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:49:27,658 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 01:49:27,658 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 01:49:27,668 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:49:27,668 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 01:49:27,668 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 01:49:27,689 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:49:27,689 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:27,690 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:27,690 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards 01:49:27,690 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:49:27,690 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards 01:49:27,690 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:49:27,690 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:27,690 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:49:27,690 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:49:27,690 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:49:27,690 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:49:27,691 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-2-shard-cars-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:49:27,691 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-2-shard-people-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:49:27,696 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:27,696 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards 01:49:27,696 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:49:27,696 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards 01:49:27,696 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:49:27,696 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:27,696 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:49:27,696 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:49:27,696 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:49:27,697 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:49:27,773 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:49:27,918 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:49:27,919 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 01:49:27,919 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 01:49:27,919 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testWriteTransactionWithMultipleShards 01:49:27,920 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:27,920 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Cluster state: [] 01:49:27,920 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:27,920 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:27,920 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 01:49:27,920 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:27,920 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 01:49:27,920 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:49:27,921 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:49:27,921 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:49:27,922 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testWriteTransactionWithMultipleShards] 01:49:27,922 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:27,922 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:49:27,922 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:49:27,922 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: became leader with frontend state for [] 01:49:27,922 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from Candidate to Leader 01:49:27,922 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:49:27,922 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:49:27,922 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:49:27,922 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from Candidate to Leader 01:49:27,922 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithMultipleShards: Closing 01:49:27,922 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:49:27,923 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:49:27,923 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:49:27,923 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:49:27,923 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testWriteTransactionWithMultipleShards status sync done false 01:49:27,923 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:49:27,923 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:27,988 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:49:27,988 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 01:49:27,988 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 01:49:27,988 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testWriteTransactionWithMultipleShards 01:49:27,989 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:27,989 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Cluster state: [] 01:49:27,989 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:27,989 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:27,989 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 01:49:27,989 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:27,989 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 01:49:27,989 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:49:27,990 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:49:27,990 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:49:27,990 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testWriteTransactionWithMultipleShards] 01:49:27,990 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:27,990 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:49:27,990 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:49:27,990 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: became leader with frontend state for [] 01:49:27,991 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from Candidate to Leader 01:49:27,991 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:49:27,991 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:27,991 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:27,991 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from Candidate to Leader 01:49:27,991 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All Shards are ready - data store testWriteTransactionWithMultipleShards is ready 01:49:27,991 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithMultipleShards: Closing 01:49:27,991 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:49:27,991 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:49:27,991 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:27,991 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:27,991 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All Shards are ready - data store testWriteTransactionWithMultipleShards is ready 01:49:27,991 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithMultipleShards status sync done false 01:49:27,992 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:49:27,992 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:28,39 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testWriteTransactionWithMultipleShards status sync done true 01:49:28,69 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-1218102381] 01:49:28,69 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: findLocalShard : people 01:49:28,69 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1996875763] 01:49:28,70 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testWriteTransactionWithMultipleShards commit rateLimit adjusted to 95.19145652890465 01:49:28,70 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,70 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-1915136545] 01:49:28,70 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 01:49:28,70 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-1915136545] 01:49:28,70 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 01:49:28,70 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying 2 components for commit 01:49:28,70 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying transaction for shard cars 01:49:28,70 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 01:49:28,70 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying transaction for shard people 01:49:28,70 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 01:49:28,70 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 canCommit 01:49:28,70 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:28,70 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards, primaryVersion=13] 01:49:28,70 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:49:28,71 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards, primaryVersion=13] 01:49:28,71 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards, primaryVersion=13] 01:49:28,71 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards, primaryVersion=13] 01:49:28,71 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] for shard cars 01:49:28,71 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] for shard people 01:49:28,71 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] found - creating WRITE_ONLY transaction context 01:49:28,71 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] found - creating WRITE_ONLY transaction context 01:49:28,71 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:49:28,71 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 01:49:28,71 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 = [ 1 ] 01:49:28,71 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 = [ 1 ] 01:49:28,71 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction called 01:49:28,71 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction called 01:49:28,71 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 sending 1 batched modifications, ready: true 01:49:28,71 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 sending 1 batched modifications, ready: true 01:49:28,71 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] 01:49:28,71 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] 01:49:28,72 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, disabling transaction tracking 01:49:28,73 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 01:49:28,73 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 01:49:28,73 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:49:28,73 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:28,73 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 01:49:28,73 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}]]] 01:49:28,73 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,73 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, disabling transaction tracking 01:49:28,73 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Readying Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 of 1 operations, client version 13 01:49:28,73 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,73 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 ready 01:49:28,73 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 01:49:28,73 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 01:49:28,73 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:49:28,73 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:28,73 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testWriteTransactionWithMultipleShards 01:49:28,74 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}]]] 01:49:28,74 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,74 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Readying Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 of 1 operations, client version 13 01:49:28,74 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,74 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 ready 01:49:28,74 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction 01:49:28,74 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-191211768)] resolved 01:49:28,74 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:49:28,74 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}]]] 01:49:28,74 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:49:28,74 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:49:28,74 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 01:49:28,74 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 01:49:28,74 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Log size is now 1 01:49:28,74 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:28,75 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:49:28,75 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:28,75 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction 01:49:28,75 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:49:28,75 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1996875763)] resolved 01:49:28,75 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: successfully resolved all cohort actors 01:49:28,75 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 finishCanCommit 01:49:28,76 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}]]] 01:49:28,76 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:49:28,76 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:49:28,76 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-191211768)] 01:49:28,76 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 01:49:28,76 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-191211768)] 01:49:28,76 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 01:49:28,76 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Log size is now 1 01:49:28,76 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 01:49:28,76 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:28,76 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithMultipleShards status sync done true 01:49:28,77 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 01:49:28,77 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:49:28,77 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 01:49:28,77 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:28,77 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:49:28,77 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 01:49:28,77 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:49:28,77 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 01:49:28,77 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 01:49:28,77 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:49:28,77 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-1-shard-cars-testWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:49:28,77 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:49:28,77 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, nanosAgo=4969361} is already disabled 01:49:28,77 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 01:49:28,77 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:49:28,77 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:49:28,77 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 01:49:28,77 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:49:28,78 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-1-shard-people-testWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:49:28,78 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Can committing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,78 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, nanosAgo=4463183} is already disabled 01:49:28,78 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,78 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:49:28,78 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 validated 01:49:28,78 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:28,78 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: canCommit for member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: success 01:49:28,79 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:49:28,79 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1996875763)] 01:49:28,79 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1996875763)] 01:49:28,79 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Can committing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,79 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,79 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 validated 01:49:28,80 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:28,80 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: canCommit for member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: success 01:49:28,80 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:49:28,80 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit returning result: true 01:49:28,80 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 commit 01:49:28,80 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 finish commit 01:49:28,80 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-191211768)] 01:49:28,80 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-191211768)] 01:49:28,80 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1996875763)] 01:49:28,81 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1996875763)] 01:49:28,81 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Committing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,81 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,81 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 candidate ready 01:49:28,81 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:49:28,81 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - messages: [] 01:49:28,81 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:49:28,81 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 prepared 01:49:28,81 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:28,81 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Finishing commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,81 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,81 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 175 01:49:28,81 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 01:49:28,81 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}]]] 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 submitted to persistence 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Committing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 candidate ready 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - messages: [] 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 prepared 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Finishing commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 179 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testWriteTransactionWithMultipleShards 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 01:49:28,82 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 submitted to persistence 01:49:28,83 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}]]] 01:49:28,83 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:49:28,83 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:49:28,83 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 01:49:28,83 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=175} 01:49:28,83 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:49:28,83 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 01:49:28,83 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:49:28,83 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=179} 01:49:28,83 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:49:28,83 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Log size is now 2 01:49:28,83 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Commit index set to 0 01:49:28,83 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Log size is now 2 01:49:28,83 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Commit index set to 0 01:49:28,83 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:28,83 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:28,83 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:49:28,83 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 01:49:28,83 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:49:28,83 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:49:28,83 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 01:49:28,83 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 01:49:28,83 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:49:28,83 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:49:28,83 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-people-testWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:49:28,83 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 01:49:28,83 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:49:28,83 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:49:28,84 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 01:49:28,84 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:49:28,84 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:49:28,84 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:49:28,84 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 01:49:28,84 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:49:28,84 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:49:28,84 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:49:28,84 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 01:49:28,84 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:49:28,84 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=175} 01:49:28,84 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,84 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:49:28,84 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:49:28,84 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testWriteTransactionWithMultipleShards_member-1-shard-cars-testWriteTransactionWithMultipleShards%23-191211768$e] 01:49:28,84 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:49:28,84 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:49:28,84 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 01:49:28,84 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:49:28,84 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=179} 01:49:28,84 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,84 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:49:28,84 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:49:28,84 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-986764270] with name member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher 01:49:28,84 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testWriteTransactionWithMultipleShards_member-1-shard-people-testWriteTransactionWithMultipleShards%23-1996875763$f] 01:49:28,84 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-986764270] 01:49:28,84 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:49:28,84 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:49:28,84 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#835540172] with name member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher 01:49:28,84 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:28,84 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:49:28,84 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-986764270] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:28,84 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#835540172] 01:49:28,84 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:49:28,85 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:28,85 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher: 61.01 μs 01:49:28,85 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:49:28,85 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:49:28,85 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#835540172] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:28,85 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher: 120.3 μs 01:49:28,88 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Commit index set to 1 01:49:28,88 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Commit index set to 1 01:49:28,88 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:49:28,88 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:49:28,88 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 01:49:28,88 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 01:49:28,88 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:49:28,88 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:49:28,88 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=175} 01:49:28,88 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=179} 01:49:28,88 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,88 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:49:28,88 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#321506044] with name member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher 01:49:28,88 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-376605282] with name member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher 01:49:28,89 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:49:28,89 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:49:28,89 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:49:28,89 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:49:28,89 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-376605282] 01:49:28,89 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit succeeded 01:49:28,89 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:49:28,89 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#321506044] 01:49:28,89 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:49:28,89 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher: 49.24 μs 01:49:28,89 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher: 57.32 μs 01:49:28,89 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-376605282] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:28,89 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:49:28,89 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] for shard cars, trying to use DirectTransactionContextWrapper 01:49:28,89 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:49:28,89 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] found - trying create transaction 01:49:28,89 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] 01:49:28,89 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Queuing TransactionOperation 01:49:28,89 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#321506044] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:28,90 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:49:28,90 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_31#1782937953] 01:49:28,91 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_31#1782937953] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:28,91 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_31#1782937953, transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0, version=13] 01:49:28,91 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:49:28,91 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_31#1782937953)] 01:49:28,92 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 ReadData operation succeeded 01:49:28,92 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] for shard people, trying to use DirectTransactionContextWrapper 01:49:28,92 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:49:28,92 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] found - trying create transaction 01:49:28,92 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] 01:49:28,92 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Queuing TransactionOperation 01:49:28,94 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:49:28,94 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_32#-1264491796] 01:49:28,95 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_32#-1264491796, transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0, version=13] 01:49:28,95 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 01:49:28,95 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_32#-1264491796)] 01:49:28,96 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 ReadData operation succeeded 01:49:28,96 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:49:28,100 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testWriteTransactionWithMultipleShards 01:49:28,100 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testWriteTransactionWithMultipleShards 01:49:28,100 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithMultipleShards - waiting on 2 shards 01:49:28,100 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onShutDown 01:49:28,100 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Initiating leader transfer 01:49:28,100 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onShutDown 01:49:28,100 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Initiating leader transfer 01:49:28,100 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5fbbfb7a 01:49:28,101 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5fbbfb7a 01:49:28,101 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 01:49:28,101 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@24b47de9 01:49:28,101 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:28,101 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@24b47de9 01:49:28,101 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 01:49:28,101 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:28,101 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:49:28,101 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:28,101 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:28,101 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:49:28,101 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:49:28,101 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:49:28,104 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards] 01:49:28,104 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards] 01:49:28,104 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:49:28,104 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:28,104 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:28,104 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:49:28,104 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:49:28,104 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:49:28,105 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:49:28,105 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:49:28,105 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:28,105 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Leader transfer complete 01:49:28,105 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:49:28,105 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:49:28,105 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:28,105 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Leader transfer complete 01:49:28,518 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 01:49:28,518 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-191211768]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:49:28,519 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testWriteTransactionWithMultipleShards 01:49:28,519 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-1-shard-people-testWriteTransactionWithMultipleShards 01:49:28,519 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader)_slicer-id-565: Closing 01:49:28,519 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader)_slicer-id-564: Closing 01:49:28,519 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:49:28,519 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:49:28,519 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithMultipleShards 01:49:28,520 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 01:49:28,520 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:49:28,520 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: shutting down 01:49:28,520 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 01:49:28,520 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 01:49:28,521 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:28,521 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:28,525 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:28,525 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:28,526 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:28,526 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:28,528 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:28,540 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 01:49:28,540 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithMultipleShards 01:49:28,540 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Closing 01:49:28,540 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-people-testWriteTransactionWithMultipleShards 01:49:28,540 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:49:28,540 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Closing 01:49:28,540 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:49:28,540 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithMultipleShards 01:49:28,541 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:28,541 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:29,533 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-1071623744681294309] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:29,552 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [5696047322523677005] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:29,555 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:29,555 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:49:29.557] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:29,564 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:29,564 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:29,567 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:29.568] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:29,578 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:29,584 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [1112483543561870033] 01:49:29,584 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:29,586 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:29,586 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:29,586 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:29,586 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:29,596 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:29,596 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:29,596 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:29,600 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:29,614 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-2042022670091660886] 01:49:29,614 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:29,616 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:29,616 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:29,623 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:29,623 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:29,634 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:29,645 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-1032730850908872938] 01:49:29,645 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:29,647 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:29,648 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:29,648 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:29,648 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:29,657 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:29,658 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:29,660 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithMultipleShards 01:49:29,665 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithMultipleShards 01:49:29,665 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#2000578461] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:29,669 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: no initial generation override, starting from 0 01:49:29,669 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:49:29,669 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@436b388c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@12846024 01:49:29,669 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3baded3b 01:49:29,669 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithMultipleShards 01:49:29,669 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:29,670 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-1-shard-people-testTransactionChainWithMultipleShards 01:49:29,670 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:29,670 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:29,670 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:49:29,670 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionChainWithMultipleShards, 0, 1710164969670, None)) 01:49:29,670 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164969669,0,0)) 01:49:29,670 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@18cc8ad8 01:49:29,670 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@12846024 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7774114c 01:49:29,670 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} started 01:49:29,671 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:29,672 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:29,673 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithMultipleShards 01:49:29,674 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:29,674 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:29,678 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:29,678 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:29,678 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-cars-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards on shard member-1-shard-cars-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-804098297] 01:49:29,678 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testTransactionChainWithMultipleShards 01:49:29,678 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionChainWithMultipleShards, persistent : true 01:49:29,678 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-804098297] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:29,679 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionChainWithMultipleShards with journal batch size 1 01:49:29,679 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Shard created : member-1-shard-people-testTransactionChainWithMultipleShards, persistent : true 01:49:29,679 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Starting recovery for member-1-shard-people-testTransactionChainWithMultipleShards with journal batch size 1 01:49:29,681 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-notifier#1373418581 created and ready for shard:member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:29,681 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#1305358868] 01:49:29,681 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1760812235] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:29,681 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-notifier#1373418581] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:29,681 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#1305358868] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:29,681 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-notifier#1593494842 created and ready for shard:member-1-shard-people-testTransactionChainWithMultipleShards 01:49:29,681 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#-1397453609] 01:49:29,681 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-notifier#1593494842] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:29,685 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:49:29,685 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:49:29,685 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-people-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards on shard member-1-shard-people-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1760812235] 01:49:29,685 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:29,685 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:29,686 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [people] 01:49:29,686 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-testTransactionChainWithMultipleShards is initialized 01:49:29,686 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:29,686 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionChainWithMultipleShards is no longer the leader 01:49:29,686 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from null to Follower 01:49:29,686 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 01:49:29,686 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from null to Follower 01:49:29,690 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#-1397453609] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:29,690 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#718321753] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:29,691 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithMultipleShards 01:49:29,693 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: no initial generation override, starting from 0 01:49:29,693 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:49:29,693 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4522be4f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@75a353d3 01:49:29,693 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@719cc9bf 01:49:29,693 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithMultipleShards 01:49:29,693 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:29,694 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-2-shard-people-testTransactionChainWithMultipleShards 01:49:29,694 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionChainWithMultipleShards, 0, 1710164969694, None)) 01:49:29,694 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164969693,0,0)) 01:49:29,694 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@75a353d3 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5600c20 01:49:29,695 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7024c331 01:49:29,695 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} started 01:49:29,695 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:29,695 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:29,697 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:29,697 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:29,697 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:29,697 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 01:49:29,697 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [cars] 01:49:29,697 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionChainWithMultipleShards is initialized 01:49:29,697 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from null to Follower 01:49:29,697 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 01:49:29,697 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from null to Follower 01:49:29,698 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testTransactionChainWithMultipleShards 01:49:29,698 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionChainWithMultipleShards, persistent : true 01:49:29,698 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards#1132736539] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:29,699 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#1796934718] 01:49:29,699 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-notifier#1939274009] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:29,699 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#1796934718] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:29,699 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-people-testTransactionChainWithMultipleShards, persistent : true 01:49:29,700 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-people-testTransactionChainWithMultipleShards with journal batch size 1 01:49:29,700 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards#725077046] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:29,700 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-1612512066] 01:49:29,700 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-notifier#1186429589 created and ready for shard:member-2-shard-people-testTransactionChainWithMultipleShards 01:49:29,700 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-notifier#1186429589] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:29,700 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-1612512066] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:29,701 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionChainWithMultipleShards with journal batch size 1 01:49:29,702 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-notifier#1939274009 created and ready for shard:member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:29,702 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:29,702 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:29,702 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [people] 01:49:29,702 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-testTransactionChainWithMultipleShards is initialized 01:49:29,702 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:29,702 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionChainWithMultipleShards is no longer the leader 01:49:29,702 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionChainWithMultipleShards , received role change from null to Follower 01:49:29,702 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 01:49:29,702 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:29,702 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:29,703 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:29,703 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 01:49:29,703 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithMultipleShards , received role change from null to Follower 01:49:29,703 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-2-shard-people-testTransactionChainWithMultipleShards from null to Follower 01:49:29,703 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [cars] 01:49:29,703 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionChainWithMultipleShards is initialized 01:49:29,703 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 01:49:29,705 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-2-shard-cars-testTransactionChainWithMultipleShards from null to Follower 01:49:29,717 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:49:29,717 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:29,719 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:29,720 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:29,721 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:29,721 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:29,721 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:49:29,721 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:49:29,721 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-1-shard-people-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:49:29,721 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:29,721 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:29,745 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#2000578461] 01:49:29,745 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: findLocalShard : cars 01:49:29,745 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-804098297] 01:49:29,936 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:29,936 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:49:29,936 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionChainWithMultipleShards] 01:49:29,936 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:29,936 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 1 01:49:29,937 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:29,937 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Closing 01:49:29,937 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:29,937 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 01:49:29,937 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from Follower to Candidate 01:49:29,937 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from Follower to Candidate 01:49:29,956 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:29,956 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:49:29,956 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testTransactionChainWithMultipleShards] 01:49:29,956 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 01:49:29,956 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 1 01:49:29,956 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:29,956 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Closing 01:49:29,956 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:29,956 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionChainWithMultipleShards is no longer the leader 01:49:29,957 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from Follower to Candidate 01:49:29,957 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from Follower to Candidate 01:49:30,207 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:49:30,207 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 01:49:30,207 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 2 01:49:30,236 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:49:30,236 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:30,236 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 2 01:49:30,249 PM [cluster-test-shard-dispatcher-31] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:30,319 PM [cluster-test-shard-dispatcher-31] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:30,467 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:49:30,467 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 01:49:30,467 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 3 01:49:30,506 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:49:30,506 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:30,506 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 3 01:49:30,597 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:49:30,597 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:30,598 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:30,598 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards 01:49:30,598 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:30,598 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards 01:49:30,598 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:49:30,598 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:30,598 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:30,598 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:30,598 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:49:30,598 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:49:30,598 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:30,598 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Peer address for peer member-2-shard-people-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:49:30,654 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:30,654 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards 01:49:30,654 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:30,654 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards 01:49:30,654 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:49:30,654 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:30,654 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:30,654 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:30,654 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:49:30,654 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:49:30,663 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:49:30,756 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:49:30,756 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 01:49:30,756 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 4 01:49:30,756 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testTransactionChainWithMultipleShards 01:49:30,758 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:30,758 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Cluster state: [] 01:49:30,758 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:30,758 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:30,758 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=null 01:49:30,758 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:30,758 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 01:49:30,758 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:49:30,763 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:49:30,763 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:49:30,763 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testTransactionChainWithMultipleShards] 01:49:30,763 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:30,764 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:49:30,764 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:49:30,764 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: became leader with frontend state for [] 01:49:30,764 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from Candidate to Leader 01:49:30,764 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 01:49:30,764 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:49:30,764 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:49:30,764 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from Candidate to Leader 01:49:30,764 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionChainWithMultipleShards: Closing 01:49:30,764 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testTransactionChainWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:49:30,764 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 01:49:30,765 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:49:30,765 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:49:30,765 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-people-testTransactionChainWithMultipleShards status sync done false 01:49:30,765 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:49:30,765 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:30,807 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:49:30,807 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:30,807 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 4 01:49:30,807 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:30,808 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:30,808 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Cluster state: [] 01:49:30,808 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:30,808 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:30,808 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=null 01:49:30,809 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:30,809 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:30,809 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:49:30,811 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:49:30,811 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:49:30,811 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionChainWithMultipleShards] 01:49:30,811 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:30,811 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:49:30,811 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:49:30,811 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: became leader with frontend state for [] 01:49:30,811 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 01:49:30,811 PM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:30,811 PM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:30,811 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from Candidate to Leader 01:49:30,812 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from Candidate to Leader 01:49:30,812 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All Shards are ready - data store testTransactionChainWithMultipleShards is ready 01:49:30,812 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithMultipleShards: Closing 01:49:30,812 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testTransactionChainWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:30,812 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 01:49:30,812 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:30,812 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:30,812 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All Shards are ready - data store testTransactionChainWithMultipleShards is ready 01:49:30,812 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-cars-testTransactionChainWithMultipleShards status sync done false 01:49:30,813 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:49:30,813 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:30,877 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-people-testTransactionChainWithMultipleShards status sync done true 01:49:30,902 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#2000578461] 01:49:30,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: findLocalShard : people 01:49:30,902 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1760812235] 01:49:30,903 PM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@53a10e13 01:49:30,903 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@53a10e13 changed state from IDLE to TX_OPEN 01:49:30,904 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} 01:49:30,904 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:49:30,904 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:49:30,904 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#718321753] 01:49:30,904 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:30,904 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} shard 1 01:49:30,904 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards, primaryVersion=13] 01:49:30,904 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:49:30,904 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:30,905 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards, primaryVersion=13] 01:49:30,905 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 01:49:30,905 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} 01:49:30,905 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:49:30,905 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:49:30,905 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#718321753] 01:49:30,905 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:49:30,905 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} shard 2 01:49:30,905 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} scheduling timeout in 30 seconds 01:49:30,905 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards, primaryVersion=13] 01:49:30,905 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:30,905 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} allocated sequence 0 01:49:30,905 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@5938c9d9, attempting to connect 01:49:30,905 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@3a323349 01:49:30,906 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@160afa63 01:49:30,906 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@53a10e13 changed state from TX_OPEN to IDLE 01:49:30,906 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@53a10e13 readied transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:30,906 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 01:49:30,906 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@2ec87c8b 01:49:30,906 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=FLUSHED} allocated sequence 1 01:49:30,906 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@755aa66b 01:49:30,907 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards, primaryVersion=13] 01:49:30,907 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@279e9552, attempting to connect 01:49:30,913 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} is not yet known 01:49:30,913 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} is not yet known 01:49:30,914 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, nanosAgo=32870, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:49:30,914 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, nanosAgo=31700, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:49:30,920 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:49:30,920 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:49:30,920 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-804098297], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:49:30,920 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-804098297], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:30,920 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2fe90278 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@1938d8b3 01:49:30,920 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:49:30,920 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-804098297], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:49:30,920 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5867a984 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717239690588} 01:49:30,920 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@1938d8b3 01:49:30,920 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:49:30,920 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} 01:49:30,920 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=2, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} 01:49:30,921 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 01:49:30,921 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2218/0x00000008016a3c10@5c47d749 01:49:30,921 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5867a984 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=2, protocol=READY}, enqueuedTicks=717240780885} 01:49:30,921 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=2, protocol=READY}} 01:49:30,921 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message size: 409 bytes, total slices to send: 1 01:49:30,921 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message does not need to be sliced - sending original message 01:49:30,921 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} 01:49:30,921 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 1 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2219/0x00000008016a6000@20a4773 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5867a984 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=717240958908} 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE}} 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message size: 289 bytes, total slices to send: 1 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message does not need to be sliced - sending original message 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@53a10e13 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-804098297], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@15ab5105 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2a98f73d superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@4ab719b2, splicing queue 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2a98f73d queue spliced 0 messages 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-804098297], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.841 ms 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1760812235], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1760812235], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@64d8de17 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@30d2a7e1 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} previous state FLUSHED 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1760812235], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d9e9fd9 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717240161275} 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@30d2a7e1 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=2, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} allocated sequence 0 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2218/0x00000008016a3c10@27b37deb 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d9e9fd9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=2, protocol=READY}, enqueuedTicks=717240828806} 01:49:30,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, message: Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=2, protocol=READY}} 01:49:30,923 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message size: 417 bytes, total slices to send: 1 01:49:30,923 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message does not need to be sliced - sending original message 01:49:30,923 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} 01:49:30,923 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 1 01:49:30,923 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2219/0x00000008016a6000@54d3f4fa 01:49:30,923 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d9e9fd9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=717240996259} 01:49:30,923 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, message: Envelope{sessionId=1, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE}} 01:49:30,923 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message size: 290 bytes, total slices to send: 1 01:49:30,923 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message does not need to be sliced - sending original message 01:49:30,923 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@53a10e13 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1760812235], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:49:30,923 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 01:49:30,923 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@71e32e41 01:49:30,923 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@49f665f2 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@40255fc9, splicing queue 01:49:30,923 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@49f665f2 queue spliced 0 messages 01:49:30,923 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1760812235], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 786.5 μs 01:49:30,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 01:49:30,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 01:49:30,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:49:30,929 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 01:49:30,929 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 01:49:30,929 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:49:30,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] LeaderFrontendState - member-1-shard-cars-testTransactionChainWithMultipleShards: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:49:30,929 PM [cluster-test-shard-dispatcher-17] [DEBUG] LeaderFrontendState - member-1-shard-people-testTransactionChainWithMultipleShards: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:49:30,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:49:30,929 PM [cluster-test-shard-dispatcher-17] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:49:30,929 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:49:30,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:49:30,929 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@3f54df30 01:49:30,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@46f5273d 01:49:30,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:49:30,930 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:49:30,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 ready 01:49:30,930 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 ready 01:49:30,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 01:49:30,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 to ready 01:49:30,930 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE, modificationType=WRITE}}}]} 01:49:30,930 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 to ready 01:49:30,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 is already in state READY (READY) 01:49:30,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 initiating canCommit 01:49:30,930 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 is already in state READY (READY) 01:49:30,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:49:30,930 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 initiating canCommit 01:49:30,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 validated 01:49:30,930 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:49:30,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:30,930 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 validated 01:49:30,930 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:30,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 completed canCommit 01:49:30,930 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 completed canCommit 01:49:30,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 01:49:30,930 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 01:49:30,931 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-cars-testTransactionChainWithMultipleShards status sync done true 01:49:30,936 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:30,936 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}]]] 01:49:30,937 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=2, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0}} 01:49:30,937 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=2, protocol=READY}, enqueuedTicks=717240780885, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0}} 01:49:30,937 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=2, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0} 01:49:30,937 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=2, protocol=READY} with Envelope{sessionId=1, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0}} 01:49:30,937 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=2, protocol=READY}, enqueuedTicks=717240828806, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0}} 01:49:30,937 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=2, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0} 01:49:30,937 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1}} 01:49:30,937 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=717240958908, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1}} 01:49:30,937 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} canCommit completed 01:49:30,937 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1}} 01:49:30,937 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=717240996259, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1}} 01:49:30,937 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} canCommit completed 01:49:30,937 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:49:30,937 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@1f415bb8 01:49:30,937 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} 01:49:30,938 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}]]] 01:49:30,938 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 01:49:30,938 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:49:30,938 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 01:49:30,938 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 01:49:30,938 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 1 01:49:30,938 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:30,938 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 2 01:49:30,938 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1242/0x0000000801303c28@52c9c92a 01:49:30,938 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5867a984 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717273246676} 01:49:30,938 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 01:49:30,938 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 2 01:49:30,938 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@475abe6a 01:49:30,938 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} 01:49:30,938 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 2 01:49:30,938 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1242/0x0000000801303c28@52d3a1a2 01:49:30,938 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d9e9fd9 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717273513700} 01:49:30,939 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:49:30,939 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:49:30,939 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 0 01:49:30,939 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 01:49:30,939 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 01:49:30,939 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2220/0x00000008016a65d8@4bea7443 01:49:30,939 PM [cluster-test-shard-dispatcher-17] [DEBUG] LeaderFrontendState - member-1-shard-cars-testTransactionChainWithMultipleShards: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:49:30,940 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 initiating preCommit 01:49:30,940 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:49:30,940 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 candidate ready 01:49:30,940 PM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2: canCommit - candidate rootPath: / 01:49:30,940 PM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2: canCommit - messages: [] 01:49:30,940 PM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2: preCommit - successfulFromPrevious: [] 01:49:30,940 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 prepared 01:49:30,940 PM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:30,940 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 completed preCommit 01:49:30,941 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionChainWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5deb9022}, purged=MutableUnsignedLongSet{size=0}} 01:49:30,941 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:49:30,941 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:30,941 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:49:30,942 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 initiating preCommit 01:49:30,942 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:49:30,942 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 candidate ready 01:49:30,942 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1: canCommit - candidate rootPath: / 01:49:30,942 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1: canCommit - messages: [] 01:49:30,942 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1: preCommit - successfulFromPrevious: [] 01:49:30,942 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 prepared 01:49:30,942 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:30,942 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 completed preCommit 01:49:30,943 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 0 01:49:30,943 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} with Envelope{sessionId=1, txSequence=3, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2}} 01:49:30,944 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717273513700, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2}} 01:49:30,944 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:49:30,944 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 0 01:49:30,944 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:49:30,944 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 01:49:30,944 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 01:49:30,944 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} 01:49:30,944 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717239690588, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} 01:49:30,944 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionChainWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@2bf2bb4}, purged=MutableUnsignedLongSet{size=0}} 01:49:30,944 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0} 01:49:30,944 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:49:30,944 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} with Envelope{sessionId=0, txSequence=3, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2}} 01:49:30,944 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717273246676, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2}} 01:49:30,944 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:49:30,944 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:49:30,944 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 3 01:49:30,944 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@3e389958 01:49:30,944 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} 01:49:30,945 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 3 01:49:30,945 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@3e389958 01:49:30,945 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-804098297], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:30,945 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5867a984 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717279942925} 01:49:30,945 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 3 01:49:30,945 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@2c83d91b 01:49:30,945 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} 01:49:30,945 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 3 01:49:30,945 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@2c83d91b 01:49:30,945 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d9e9fd9 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717280296290} 01:49:30,947 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 initiating commit 01:49:30,947 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 initiating commit 01:49:30,948 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:49:30,948 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:49:30,948 PM [cluster-test-shard-dispatcher-17] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 177 01:49:30,948 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 01:49:30,948 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 01:49:30,948 PM [cluster-test-shard-dispatcher-19] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 181 01:49:30,948 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:30,948 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 01:49:30,948 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 01:49:30,948 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:30,949 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:30,949 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}]]] 01:49:30,949 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:30,949 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 submitted to persistence 01:49:30,949 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:30,949 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 01:49:30,949 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}]]] 01:49:30,949 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 submitted to persistence 01:49:30,950 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 01:49:30,950 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 01:49:30,951 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}]]] 01:49:30,951 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 01:49:30,951 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:49:30,951 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=177} 01:49:30,951 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 01:49:30,951 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 2 01:49:30,951 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:30,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}]]] 01:49:30,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 01:49:30,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:49:30,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 01:49:30,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 01:49:30,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=181} 01:49:30,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 01:49:30,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 2 01:49:30,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:30,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 01:49:30,951 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 01:49:30,951 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 01:49:30,951 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:49:30,952 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:49:30,952 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 01:49:30,952 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:49:30,952 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 0 01:49:30,952 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 1 01:49:30,952 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 01:49:30,952 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 01:49:30,952 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 1 data Simple{size=177} 01:49:30,952 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 01:49:30,952 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:49:30,952 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2220/0x00000008016a65d8@4c16b5f8 01:49:30,952 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:49:30,952 PM [cluster-test-shard-dispatcher-19] [DEBUG] LeaderFrontendState - member-1-shard-people-testTransactionChainWithMultipleShards: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:49:30,952 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1: commit - successfulFromPrevious: [] 01:49:30,952 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:49:30,952 PM [cluster-test-shard-dispatcher-17] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 01:49:30,952 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionChainWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4b2571b0}, purged=MutableUnsignedLongSet{size=0}} 01:49:30,952 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:49:30,952 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 1 01:49:30,952 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 01:49:30,952 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 1 data Simple{size=181} 01:49:30,952 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:49:30,952 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:49:30,952 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#448517410] with name member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher 01:49:30,952 PM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2: commit - successfulFromPrevious: [] 01:49:30,952 PM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 01:49:30,952 PM [cluster-test-shard-dispatcher-19] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE, modificationType=WRITE}}}]} 01:49:30,952 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:49:30,952 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:30,952 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-928251525] with name member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher 01:49:30,952 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:49:30,952 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-928251525] 01:49:30,952 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:49:30,952 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:30,952 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:49:30,952 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:49:30,953 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-928251525] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:30,953 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#448517410] 01:49:30,953 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 92.57 μs 01:49:30,953 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:49:30,953 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#448517410] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:30,953 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 105.3 μs 01:49:30,955 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 1 01:49:30,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 1 01:49:30,955 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:49:30,955 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 1 01:49:30,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 01:49:30,955 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 01:49:30,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 0 01:49:30,955 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 1 data Simple{size=177} 01:49:30,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} with Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} 01:49:30,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717240161275, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0} 01:49:30,955 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} with Envelope{sessionId=0, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3}} 01:49:30,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionChainWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@39f4124b}, purged=MutableUnsignedLongSet{size=0}} 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717279942925, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3}} 01:49:30,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:49:30,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 1 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} doCommit completed 01:49:30,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2fe90278 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 01:49:30,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 1 data Simple{size=181} 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: initiating purge 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 4 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@11787b9a 01:49:30,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} 01:49:30,955 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:49:30,955 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1403585701] with name member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher 01:49:30,955 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:49:30,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:49:30,955 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED}: initiating purge 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 4 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@322ede5 01:49:30,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#1730851024] with name member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-804098297], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5867a984 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717290079141} 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} with Envelope{sessionId=1, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3}} 01:49:30,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717280296290, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3}} 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} doCommit completed 01:49:30,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@64d8de17 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR}: initiating purge 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 4 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@3b65235c 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED}: initiating purge 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} allocated sequence 4 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@4ce50dac 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1403585701] 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1760812235], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d9e9fd9 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717290639250} 01:49:30,955 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#1730851024] 01:49:30,956 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:49:30,956 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:49:30,956 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@53a10e13 changed state from IDLE to TX_OPEN 01:49:30,956 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 44.28 μs 01:49:30,956 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1403585701] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:30,956 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#1730851024] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:30,956 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:30,956 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 0 01:49:30,956 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:30,956 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} allocated sequence 0 01:49:30,956 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 1 01:49:30,956 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@10d75564 01:49:30,956 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5867a984 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=1}, enqueuedTicks=717291394532} 01:49:30,956 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=6, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=1}} 01:49:30,956 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message size: 453 bytes, total slices to send: 1 01:49:30,956 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message does not need to be sliced - sending original message 01:49:30,956 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@32a89d72 01:49:30,956 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5867a984 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]}, enqueuedTicks=717291727017} 01:49:30,957 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 61.63 μs 01:49:30,962 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:49:30,962 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 01:49:30,962 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 01:49:30,962 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:49:30,963 PM [cluster-test-shard-dispatcher-17] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:49:30,963 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:49:30,963 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@455c70a0 01:49:30,963 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:49:30,963 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 01:49:30,963 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 01:49:30,964 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:49:30,964 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-567: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1}} 01:49:30,964 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 01:49:30,964 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 01:49:30,965 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-567: Message size: 371 bytes, total slices to send: 1 01:49:30,965 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-567: Message does not need to be sliced - sending original message 01:49:30,965 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=1} with Envelope{sessionId=0, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0}} 01:49:30,965 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=1}, enqueuedTicks=717291394532, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0}} 01:49:30,965 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=1} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0} 01:49:30,966 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]} with Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1}} 01:49:30,966 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]}, enqueuedTicks=717291727017, sessionId=0, txSequence=7} with Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1}} 01:49:30,966 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1} 01:49:30,966 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} allocated sequence 1 01:49:30,966 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@33107007 01:49:30,966 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d9e9fd9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=1}, enqueuedTicks=717301696290} 01:49:30,966 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, message: Envelope{sessionId=1, txSequence=6, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=1}} 01:49:30,967 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message size: 441 bytes, total slices to send: 1 01:49:30,967 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message does not need to be sliced - sending original message 01:49:30,967 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@20ae8020 01:49:30,967 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d9e9fd9 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]}, enqueuedTicks=717302035426} 01:49:30,969 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:49:30,969 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:49:30,969 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@3231a9d9 01:49:30,969 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-568: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, message: Envelope{sessionId=1, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1}} 01:49:30,970 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SlicedMessageState - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-568: Message size: 359 bytes, total slices to send: 1 01:49:30,970 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-568: Message does not need to be sliced - sending original message 01:49:30,971 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=1} with Envelope{sessionId=1, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0}} 01:49:30,971 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=1}, enqueuedTicks=717301696290, sessionId=1, txSequence=6} with Envelope{sessionId=1, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0}} 01:49:30,971 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=1} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0} 01:49:30,971 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]} with Envelope{sessionId=1, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1}} 01:49:30,971 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]}, enqueuedTicks=717302035426, sessionId=1, txSequence=7} with Envelope{sessionId=1, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1}} 01:49:30,971 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1} 01:49:30,971 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 2 01:49:30,971 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@512f7125 01:49:30,971 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5867a984 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=READY}, enqueuedTicks=717306367637} 01:49:30,971 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=READY}} 01:49:30,971 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message size: 290 bytes, total slices to send: 1 01:49:30,971 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message does not need to be sliced - sending original message 01:49:30,971 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} allocated sequence 2 01:49:30,971 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@181c5740 01:49:30,971 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d9e9fd9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=READY}, enqueuedTicks=717306588190} 01:49:30,971 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, message: Envelope{sessionId=1, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=READY}} 01:49:30,971 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message size: 291 bytes, total slices to send: 1 01:49:30,971 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message does not need to be sliced - sending original message 01:49:30,971 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@53a10e13 changed state from TX_OPEN to IDLE 01:49:30,971 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@53a10e13 readied transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:30,971 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@53a10e13 changed state from IDLE to TX_OPEN 01:49:30,972 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:49:30,972 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=OPEN} allocated sequence 0 01:49:30,972 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@76e7240a 01:49:30,972 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d9e9fd9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=1, protocol=READY}, enqueuedTicks=717306920226} 01:49:30,972 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, message: Envelope{sessionId=1, txSequence=9, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=1, protocol=READY}} 01:49:30,972 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message size: 427 bytes, total slices to send: 1 01:49:30,972 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message does not need to be sliced - sending original message 01:49:30,972 PM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:49:30,972 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@53a10e13 changed state from TX_OPEN to IDLE 01:49:30,972 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@53a10e13 readied transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:49:30,972 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} allocated sequence 3 01:49:30,972 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@7372ad99 01:49:30,975 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:49:30,975 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 ready 01:49:30,975 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:49:30,975 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 to ready 01:49:30,975 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:49:30,975 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 ready 01:49:30,975 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}]} 01:49:30,975 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 to ready 01:49:30,976 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:49:30,976 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Reusing a chained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:49:30,976 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 snapshot MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}] 01:49:30,976 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:49:30,976 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 ready 01:49:30,976 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=DELETE}}}}}}}]} 01:49:30,976 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 to ready 01:49:30,976 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@1938d8b3 skipping transactions [2] 01:49:30,977 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=READY} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2}} 01:49:30,977 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=READY}, enqueuedTicks=717306367637, sessionId=0, txSequence=8} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2}} 01:49:30,977 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2} 01:49:30,977 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=READY} with Envelope{sessionId=1, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2}} 01:49:30,977 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=READY}, enqueuedTicks=717306588190, sessionId=1, txSequence=8} with Envelope{sessionId=1, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2}} 01:49:30,977 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2} 01:49:30,977 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=1, protocol=READY} with Envelope{sessionId=1, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0}} 01:49:30,977 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=1, protocol=READY}, enqueuedTicks=717306920226, sessionId=1, txSequence=9} with Envelope{sessionId=1, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0}} 01:49:30,977 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0} 01:49:30,978 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5867a984 transmitting entry ConnectionEntry{request=SkipTransactionsRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717312884623} 01:49:30,978 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5867a984 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=717312970814} 01:49:30,978 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=a, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE}} 01:49:30,978 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message size: 290 bytes, total slices to send: 1 01:49:30,978 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message does not need to be sliced - sending original message 01:49:30,978 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} allocated sequence 3 01:49:30,978 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@250bc1a6 01:49:30,978 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d9e9fd9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=717313175378} 01:49:30,978 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, message: Envelope{sessionId=1, txSequence=a, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE}} 01:49:30,978 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message size: 291 bytes, total slices to send: 1 01:49:30,978 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message does not need to be sliced - sending original message 01:49:30,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 is already in state READY (READY) 01:49:30,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 initiating canCommit 01:49:30,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:49:30,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 validated 01:49:30,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:30,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 completed canCommit 01:49:30,982 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} skipping transactions [[2..2]] 01:49:30,983 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 01:49:30,983 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 01:49:30,984 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.SkipTransactionsPayload, isSendImmediate: false 01:49:30,984 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 is already in state READY (READY) 01:49:30,984 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 initiating canCommit 01:49:30,984 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:49:30,984 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 validated 01:49:30,984 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:30,984 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 completed canCommit 01:49:30,984 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 01:49:30,984 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=a, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3}} 01:49:30,984 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=717313175378, sessionId=1, txSequence=10} with Envelope{sessionId=1, txSequence=a, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3}} 01:49:30,985 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} canCommit completed 01:49:30,985 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=a, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3}} 01:49:30,985 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=717312970814, sessionId=0, txSequence=10} with Envelope{sessionId=0, txSequence=a, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3}} 01:49:30,985 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} canCommit completed 01:49:30,985 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} allocated sequence 4 01:49:30,985 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@313bbdca 01:49:30,985 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5867a984 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717320011949} 01:49:30,985 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} allocated sequence 4 01:49:30,985 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@269deb29 01:49:30,985 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d9e9fd9 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717320087340} 01:49:30,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 initiating preCommit 01:49:30,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:49:30,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 candidate ready 01:49:30,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1: canCommit - candidate rootPath: / 01:49:30,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1: canCommit - messages: [] 01:49:30,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1: preCommit - successfulFromPrevious: [] 01:49:30,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 prepared 01:49:30,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:30,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 completed preCommit 01:49:30,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 initiating preCommit 01:49:30,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:49:30,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 candidate ready 01:49:30,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2: canCommit - candidate rootPath: / 01:49:30,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2: canCommit - messages: [] 01:49:30,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2: preCommit - successfulFromPrevious: [] 01:49:30,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 prepared 01:49:30,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:30,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 completed preCommit 01:49:30,987 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} with Envelope{sessionId=0, txSequence=b, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4}} 01:49:30,987 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717320011949, sessionId=0, txSequence=11} with Envelope{sessionId=0, txSequence=b, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4}} 01:49:30,987 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} preCommit completed, clearing successfulRequests 01:49:30,987 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} with Envelope{sessionId=1, txSequence=b, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4}} 01:49:30,987 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717320087340, sessionId=1, txSequence=11} with Envelope{sessionId=1, txSequence=b, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4}} 01:49:30,987 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} preCommit completed, clearing successfulRequests 01:49:30,987 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} allocated sequence 5 01:49:30,987 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@3cd72d3c 01:49:30,987 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5867a984 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717322066723} 01:49:30,987 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} allocated sequence 5 01:49:30,987 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@506e0281 01:49:30,987 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d9e9fd9 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717322134954} 01:49:30,987 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 initiating commit 01:49:30,987 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:49:30,988 PM [cluster-test-shard-dispatcher-17] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 236 01:49:30,988 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 01:49:30,988 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 01:49:30,988 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:30,988 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: 0 01:49:30,988 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:30,988 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}], SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}], SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}]]] 01:49:30,988 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 submitted to persistence 01:49:30,988 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 initiating commit 01:49:30,988 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:49:30,988 PM [cluster-test-shard-dispatcher-17] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 225 01:49:30,988 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 01:49:30,988 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 01:49:30,988 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:30,988 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 01:49:30,988 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 01:49:30,988 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}]]] 01:49:30,988 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 submitted to persistence 01:49:30,989 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 01:49:30,989 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 01:49:30,989 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}], SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}], SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}]]] 01:49:30,989 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}]]] 01:49:30,989 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 3 01:49:30,989 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 01:49:30,990 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:49:30,990 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:49:30,990 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69} 01:49:30,990 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69} 01:49:30,990 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 01:49:30,990 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 01:49:30,990 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=225} 01:49:30,990 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 01:49:30,990 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 3 01:49:30,990 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75} 01:49:30,990 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:30,990 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 01:49:30,990 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=236} 01:49:30,990 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 01:49:30,990 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 4 01:49:30,990 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:30,990 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 01:49:30,990 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 01:49:30,990 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 01:49:30,990 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 01:49:30,990 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 01:49:30,990 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 3, nextIndex: 4 01:49:30,990 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 01:49:30,991 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 2 01:49:30,991 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 01:49:30,991 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69} 01:49:30,991 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@4d05d407 01:49:30,991 PM [cluster-test-shard-dispatcher-17] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:49:30,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 4, nextIndex: 5 01:49:30,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 1 01:49:30,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 2 01:49:30,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 01:49:30,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69} 01:49:30,991 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:49:30,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@7c7345ca 01:49:30,991 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 3 01:49:30,991 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 01:49:30,991 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=225} 01:49:30,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:49:30,991 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:49:30,991 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:49:30,991 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2: commit - successfulFromPrevious: [] 01:49:30,991 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 changing state from COMMIT_PENDING to COMMITTED 01:49:30,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:49:30,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 3 01:49:30,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 01:49:30,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 3 data SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75} 01:49:30,991 PM [cluster-test-shard-dispatcher-17] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}]} 01:49:30,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2234/0x00000008016af528@1fa72a02 01:49:30,991 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:49:30,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: History LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} skipped transactions ImmutableUnsignedLongSet{span=[2..2], size=1} 01:49:30,991 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 2 for term 4 01:49:30,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 4 01:49:30,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 01:49:30,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 4 data Simple{size=236} 01:49:30,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:49:30,991 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:49:30,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:49:30,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1: commit - successfulFromPrevious: [] 01:49:30,991 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=3 01:49:30,991 PM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 01:49:30,991 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 87.48 μs 01:49:30,992 PM [cluster-test-shard-dispatcher-19] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:49:30,992 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:49:30,992 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 39.26 μs 01:49:30,992 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 4 01:49:30,992 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:49:30,992 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 01:49:30,996 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4}} 01:49:30,996 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717290639250, sessionId=1, txSequence=5} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4}} 01:49:30,996 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE}: purge completed 01:49:30,996 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@30d2a7e1 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} 01:49:30,996 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR}: purge completed 01:49:30,996 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@64d8de17 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 01:49:30,996 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} with Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4}} 01:49:30,996 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717290079141, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4}} 01:49:30,996 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 01:49:30,996 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@1938d8b3 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} 01:49:30,996 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: purge completed 01:49:30,996 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2fe90278 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 01:49:30,996 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} with Envelope{sessionId=1, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5}} 01:49:30,996 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717322134954, sessionId=1, txSequence=12} with Envelope{sessionId=1, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5}} 01:49:30,996 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} doCommit completed 01:49:30,996 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@30d2a7e1 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} 01:49:30,996 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED}: initiating purge 01:49:30,996 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=DONE} allocated sequence 6 01:49:30,996 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@4df53e1e 01:49:30,997 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1760812235], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:49:30,997 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d9e9fd9 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717331742891} 01:49:30,997 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request SkipTransactionsRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} with Envelope{sessionId=0, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0}} 01:49:30,997 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=SkipTransactionsRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717312884623, sessionId=0, txSequence=9} with Envelope{sessionId=0, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0}} 01:49:30,997 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@1938d8b3 confirmed transaction skip 01:49:30,997 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} with Envelope{sessionId=0, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5}} 01:49:30,997 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717322066723, sessionId=0, txSequence=12} with Envelope{sessionId=0, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5}} 01:49:30,997 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 3 01:49:30,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 4 01:49:30,997 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} doCommit completed 01:49:30,997 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 01:49:30,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 1, lastIndex(): 4 01:49:30,997 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@1938d8b3 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} 01:49:30,997 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 2 01:49:30,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 2 01:49:30,997 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 01:49:30,997 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED}: initiating purge 01:49:30,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 01:49:30,997 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=DONE} allocated sequence 6 01:49:30,997 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69} 01:49:30,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69} 01:49:30,997 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@cc83425 01:49:30,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:49:30,997 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-804098297], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:30,997 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:49:30,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 3 01:49:30,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 01:49:30,997 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5867a984 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717332158218} 01:49:30,997 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 3 01:49:30,997 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 01:49:30,997 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=225} 01:49:30,997 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=FLUSHED} allocated sequence 1 01:49:30,997 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@25599a41 01:49:30,997 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d9e9fd9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=SIMPLE}, enqueuedTicks=717332347641} 01:49:30,997 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, message: Envelope{sessionId=1, txSequence=e, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=SIMPLE}} 01:49:30,997 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:49:30,997 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message size: 291 bytes, total slices to send: 1 01:49:30,997 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-569: Message does not need to be sliced - sending original message 01:49:30,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 3 data SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75} 01:49:30,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: History LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} skipped transactions ImmutableUnsignedLongSet{span=[2..2], size=1} 01:49:30,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 4 01:49:30,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 01:49:30,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 4 data Simple{size=236} 01:49:30,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:49:30,998 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:49:30,998 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 2 for term 4 01:49:30,998 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=3 01:49:30,998 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:49:30,998 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 4 01:49:30,998 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 01:49:30,998 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:49:30,998 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 41.81 μs 01:49:30,998 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:49:30,998 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 26.84 μs 01:49:31,0 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:49:31,0 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:49:31,0 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 01:49:31,0 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 01:49:31,0 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 01:49:31,0 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 01:49:31,0 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:49:31,0 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:49:31,1 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 is already in state READY (READY) 01:49:31,1 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 initiating direct canCommit 01:49:31,1 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:49:31,1 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 validated 01:49:31,1 PM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:31,1 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 initiating direct preCommit 01:49:31,1 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 01:49:31,1 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:49:31,2 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 candidate ready 01:49:31,2 PM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2: canCommit - candidate rootPath: / 01:49:31,2 PM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2: canCommit - messages: [] 01:49:31,2 PM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2: preCommit - successfulFromPrevious: [] 01:49:31,2 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 prepared 01:49:31,2 PM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:31,2 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 initiating direct commit 01:49:31,2 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:49:31,2 PM [cluster-test-shard-dispatcher-19] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:49:31,2 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 01:49:31,2 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 01:49:31,2 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:31,2 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 5, leaderSnapShotIndex: 2 01:49:31,2 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 4 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 01:49:31,2 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}], SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}]]] 01:49:31,2 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 submitted to persistence 01:49:31,3 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 01:49:31,3 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 01:49:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}], SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}]]] 01:49:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 01:49:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 01:49:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70} 01:49:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 01:49:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=222} 01:49:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 01:49:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 3 01:49:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=5, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 01:49:31,5 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 01:49:31,6 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 5, nextIndex: 6 01:49:31,6 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 3 01:49:31,6 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 4 01:49:31,6 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 01:49:31,6 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70} 01:49:31,6 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@27a663c8 01:49:31,6 PM [cluster-test-shard-dispatcher-17] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:49:31,6 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:49:31,6 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 5 01:49:31,6 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 4 to 5 01:49:31,6 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 5 data Simple{size=222} 01:49:31,6 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:49:31,6 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:49:31,6 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2: commit - successfulFromPrevious: [] 01:49:31,6 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 changing state from COMMIT_PENDING to COMMITTED 01:49:31,6 PM [cluster-test-shard-dispatcher-17] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, modificationType=DISAPPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=DELETE, modificationType=DELETE}}}}}}}]} 01:49:31,6 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:49:31,6 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 4 for term 4 01:49:31,6 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 01:49:31,6 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 73.32 μs 01:49:31,6 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=5 01:49:31,7 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} with Envelope{sessionId=1, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6}} 01:49:31,7 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717331742891, sessionId=1, txSequence=13} with Envelope{sessionId=1, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6}} 01:49:31,7 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=DONE}: purge completed 01:49:31,7 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@30d2a7e1 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=DONE} 01:49:31,7 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=SIMPLE} with Envelope{sessionId=1, txSequence=e, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1}} 01:49:31,7 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=SIMPLE}, enqueuedTicks=717332347641, sessionId=1, txSequence=14} with Envelope{sessionId=1, txSequence=e, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1}} 01:49:31,7 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 5 01:49:31,7 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=FLUSHED} directCommit completed 01:49:31,7 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=FLUSHED}: initiating purge 01:49:31,7 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 3, lastIndex(): 5 01:49:31,7 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=DONE} allocated sequence 2 01:49:31,7 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 4 01:49:31,7 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 01:49:31,7 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@596fdf3c 01:49:31,7 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70} 01:49:31,7 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1760812235], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:49:31,7 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:49:31,7 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 5 01:49:31,7 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d9e9fd9 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717342357374} 01:49:31,7 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 4 to 5 01:49:31,7 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 5 data Simple{size=222} 01:49:31,7 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:49:31,7 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:49:31,7 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 4 for term 4 01:49:31,7 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=5 01:49:31,8 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:49:31,8 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 40.12 μs 01:49:31,8 PM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:49:31,8 PM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:31,8 PM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:31,8 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@1938d8b3 invoking destroy 01:49:31,8 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5867a984 transmitting entry ConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717343161888} 01:49:31,8 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@30d2a7e1 invoking destroy 01:49:31,8 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d9e9fd9 transmitting entry ConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717343267939} 01:49:31,8 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@53a10e13 changed state from IDLE to CLOSED 01:49:31,8 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5e3b271a: creating a new snapshot member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 01:49:31,8 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0} shard 1 01:49:31,8 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 01:49:31,8 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:49:31,8 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@7a87b5da 01:49:31,8 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5867a984 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=717343625435} 01:49:31,9 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:49:31,9 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 01:49:31,9 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 01:49:31,9 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:49:31,11 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 01:49:31,11 PM [cluster-test-shard-dispatcher-17] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: closing history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:49:31,12 PM [cluster-test-shard-dispatcher-19] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: closing history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:49:31,13 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:49:31,13 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 01:49:31,13 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 01:49:31,13 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.CloseLocalHistoryPayload, isSendImmediate: false 01:49:31,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:49:31,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 01:49:31,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 01:49:31,13 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 01:49:31,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CloseLocalHistoryPayload, isSendImmediate: false 01:49:31,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: allocating new open snapshot member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1 01:49:31,13 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-567: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=f, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0}} 01:49:31,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 01:49:31,14 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-567: Message size: 370 bytes, total slices to send: 1 01:49:31,14 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-567: Message does not need to be sliced - sending original message 01:49:31,14 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=f, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0}} 01:49:31,14 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=717343625435, sessionId=0, txSequence=15} with Envelope{sessionId=0, txSequence=f, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0}} 01:49:31,14 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0} 01:49:31,14 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0} shard 2 01:49:31,15 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 01:49:31,15 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:49:31,15 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@4f9621a5 01:49:31,15 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d9e9fd9 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]}, enqueuedTicks=717349858847} 01:49:31,15 PM [cluster-test-shard-dispatcher-19] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: allocating new open snapshot member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2 01:49:31,15 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-568: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=11, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0}} 01:49:31,15 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SlicedMessageState - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-568: Message size: 231 bytes, total slices to send: 1 01:49:31,15 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-568: Message does not need to be sliced - sending original message 01:49:31,16 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]} with Envelope{sessionId=1, txSequence=11, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0}} 01:49:31,16 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2017069439], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]}, enqueuedTicks=717349858847, sessionId=1, txSequence=17} with Envelope{sessionId=1, txSequence=11, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0}} 01:49:31,16 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0} 01:49:31,16 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:49:31,16 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:31,17 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testTransactionChainWithMultipleShards 01:49:31,17 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithMultipleShards - waiting on 2 shards 01:49:31,17 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onShutDown 01:49:31,17 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Initiating leader transfer 01:49:31,17 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onShutDown 01:49:31,17 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Initiating leader transfer 01:49:31,17 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3b3633bf 01:49:31,17 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@36920ad6 01:49:31,19 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:49:31,19 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:49:31,19 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3b3633bf 01:49:31,19 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@36920ad6 01:49:31,19 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Attempting to transfer leadership 01:49:31,19 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Attempting to transfer leadership 01:49:31,19 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 6, leaderSnapShotIndex: 3 01:49:31,19 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 6, leaderLastIndex: 7, leaderSnapShotIndex: 4 01:49:31,19 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:31,19 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 6 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 01:49:31,19 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}], SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}]]] 01:49:31,19 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}], SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}]]] 01:49:31,19 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:49:31,19 PM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:31,19 PM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:31,19 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:49:31,19 PM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:49:31,19 PM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:49:31,20 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards] 01:49:31,20 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:49:31,20 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:31,20 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards] 01:49:31,20 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:49:31,20 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5867a984 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@675df715, splicing queue 01:49:31,21 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-804098297], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:49:31,21 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5867a984 queue spliced 2 messages 01:49:31,21 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-804098297], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-804098297], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:31,23 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: refreshing backend for shard 1 01:49:31,23 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-804098297], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:49:31,23 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:49:31,23 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:49:31,23 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#718321753] 01:49:31,23 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}], SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}]]] 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70} 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 3 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}], SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}]]] 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 5, entries to be added from: 0 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70} 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 3 01:49:31,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=7, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:31,24 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 01:49:31,24 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 01:49:31,24 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 01:49:31,24 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d9e9fd9 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@3d0e1fc1, splicing queue 01:49:31,24 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 01:49:31,24 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 01:49:31,24 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1760812235], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:49:31,25 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d9e9fd9 queue spliced 2 messages 01:49:31,25 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1760812235], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1760812235], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:49:31,25 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: refreshing backend for shard 2 01:49:31,25 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1760812235], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:49:31,25 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:49:31,25 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:49:31,25 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#718321753] 01:49:31,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:49:31,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:49:31,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 6, nextIndex: 7 01:49:31,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 4 01:49:31,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 5 01:49:31,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 4 to 5 01:49:31,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70} 01:49:31,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@3b94397a 01:49:31,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:49:31,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:49:31,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 6 01:49:31,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 5 to 6 01:49:31,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 6 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 01:49:31,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2235/0x00000008016b63a0@1066336 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 5 for term 4 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionChainWithMultipleShards, matchIndex: 6, lastIndex: 6, isVoting: true 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Leader transfer complete 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=6 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 7, nextIndex: 8 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 5 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 6 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 5 to 6 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 6 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70} 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@6422a29 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 7 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 6 to 7 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 7 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2235/0x00000008016b63a0@3eda6b57 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 6 for term 4 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testTransactionChainWithMultipleShards, matchIndex: 7, lastIndex: 7, isVoting: true 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Leader transfer complete 01:49:31,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=7 01:49:31,27 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6}} not found in inflight queue, checking pending queue 01:49:31,27 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} with Envelope{sessionId=0, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6}} 01:49:31,27 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717332158218, sessionId=0, txSequence=13} with Envelope{sessionId=0, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6}} 01:49:31,27 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=DONE}: purge completed 01:49:31,27 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@1938d8b3 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=DONE} 01:49:31,27 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=e, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1}} not found in inflight queue, checking pending queue 01:49:31,27 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} with Envelope{sessionId=0, txSequence=e, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1}} 01:49:31,27 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717343161888, sessionId=0, txSequence=14} with Envelope{sessionId=0, txSequence=e, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1}} 01:49:31,27 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@1938d8b3 destroy completed with LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1} 01:49:31,27 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 6 01:49:31,27 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@53a10e13: removed destroyed proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@1938d8b3 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 4, lastIndex(): 6 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 5 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 4 to 5 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70} 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 6 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 5 to 6 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 6 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 5 for term 4 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=6 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 7 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 7, context.getLastApplied(): 5, lastIndex(): 7 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 6 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 5 to 6 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 6 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70} 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 7 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 6 to 7 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 7 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 6 for term 4 01:49:31,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=7 01:49:31,29 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-804098297], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:31,29 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-804098297], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:31,29 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=f, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2}} not found in inflight queue, checking pending queue 01:49:31,29 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} with Envelope{sessionId=1, txSequence=f, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2}} 01:49:31,29 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717342357374, sessionId=1, txSequence=15} with Envelope{sessionId=1, txSequence=f, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2}} 01:49:31,29 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=DONE}: purge completed 01:49:31,29 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@30d2a7e1 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=DONE} 01:49:31,29 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=10, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1}} not found in inflight queue, checking pending queue 01:49:31,29 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439]} with Envelope{sessionId=1, txSequence=10, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1}} 01:49:31,29 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439]}, enqueuedTicks=717343267939, sessionId=1, txSequence=16} with Envelope{sessionId=1, txSequence=10, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1}} 01:49:31,29 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@30d2a7e1 destroy completed with LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1} 01:49:31,29 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@53a10e13: removed destroyed proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@30d2a7e1 01:49:31,29 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1760812235], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:49:31,29 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1760812235], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:49:31,238 PM [cluster-test-shard-dispatcher-21] [WARN] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=7, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 208, lastApplied : 7, commitIndex : 7 01:49:31,437 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All shards shutdown - sending PoisonPill to self 01:49:31,437 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] ShardManager - shard-manager-testTransactionChainWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-804098297]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:49:31,437 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-people-testTransactionChainWithMultipleShards 01:49:31,437 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards (Leader)_slicer-id-572: Closing 01:49:31,438 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 01:49:31,438 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:31,438 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader)_slicer-id-573: Closing 01:49:31,438 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 01:49:31,438 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithMultipleShards 01:49:31,438 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 01:49:31,438 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:49:31,438 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: shutting down 01:49:31,439 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 01:49:31,439 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 01:49:31,439 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:31,439 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:31,444 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:31,444 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:31,444 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:31,444 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:31,446 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:31,457 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithMultipleShards: Closing 01:49:31,458 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:31,458 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Closing 01:49:31,458 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-2-shard-people-testTransactionChainWithMultipleShards 01:49:31,458 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Closing 01:49:31,458 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 01:49:31,458 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 01:49:31,458 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithMultipleShards 01:49:31,458 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:31,458 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:32,453 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [1112483543561870033] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:32,471 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:32,471 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:32,473 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-2042022670091660886] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:32,473 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:32.474] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:32,480 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:32,480 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:32,482 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:32.483] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:32,500 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:32,505 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [6205929076642694090] 01:49:32,505 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:32,512 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:32,512 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:32,512 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:32,512 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:32,517 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:32,517 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:32,517 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:32,518 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:32,532 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [800901658721741376] 01:49:32,533 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:32,534 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:32,534 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:32,544 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:32,544 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:32,562 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:32,572 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [5481524500172600172] 01:49:32,572 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:32,584 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:32,584 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:32,584 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:32,584 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:32,586 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:32,588 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:32,589 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testInstallSnapshot 01:49:32,592 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:32,595 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#2132711289] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:32,596 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:32,597 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testInstallSnapshot 01:49:32,598 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: no initial generation override, starting from 0 01:49:32,598 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 01:49:32,598 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@53be999b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1c81b9f4 01:49:32,598 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testInstallSnapshot 01:49:32,598 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Creating local shard: member-1-shard-cars-testInstallSnapshot 01:49:32,599 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:32,599 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:49:32,599 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@59833400 01:49:32,599 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testInstallSnapshot, 0, 1710164972598, None)) 01:49:32,599 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164972597,0,0)) 01:49:32,599 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1c81b9f4 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@dd40449 01:49:32,600 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} started 01:49:32,600 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:32,601 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@743590e 01:49:32,601 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:32,601 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testInstallSnapshot 01:49:32,601 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:49:32,601 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:49:32,601 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: updated peer member-1-shard-cars-testInstallSnapshot on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot on shard member-1-shard-cars-testInstallSnapshot whose actor address is Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1999229691] 01:49:32,601 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-1-shard-cars-testInstallSnapshot, persistent : true 01:49:32,601 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:32,602 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-1-shard-cars-testInstallSnapshot with journal batch size 1 01:49:32,602 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1999229691] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:32,603 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-notifier#-303153901] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:32,603 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testInstallSnapshot 01:49:32,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.SnapshotOffer from Actor[akka://cluster-test/system/in-memory-snapshot-store#966557587] 01:49:32,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: SnapshotOffer called. 01:49:32,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Moving last applied index from -1 to 5 01:49:32,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=1, votedFor=null 01:49:32,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Applying recovered snapshot 01:49:32,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-cars-testInstallSnapshot: clearing clients {} 01:49:32,605 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testInstallSnapshot: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#-92850806] with name member-1-shard-cars-testInstallSnapshot-DTCL-publisher 01:49:32,605 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#-92850806] 01:49:32,605 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testInstallSnapshot: state snapshot applied in 522.8 μs 01:49:32,605 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testInstallSnapshot: publishChanges: rootPath: / 01:49:32,605 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Recovery snapshot applied for member-1-shard-cars-testInstallSnapshot in 679.4 μs: snapshotIndex=5, snapshotTerm=1, journal-size=0 01:49:32,605 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testInstallSnapshot: Elapsed time for generation of change events for member-1-shard-cars-testInstallSnapshot-DTCL-publisher: 93.99 μs 01:49:32,605 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#-92850806] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:32,605 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/shard-member-1:snapshot-read#-1016039639] 01:49:32,606 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/shard-member-1:snapshot-read#-1016039639] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:32,606 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-notifier#-303153901 created and ready for shard:member-1-shard-cars-testInstallSnapshot 01:49:32,607 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:32,607 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Recovery completed in 3.018 ms - Switching actor to Follower - last log index = 5, last log term = 1, snapshot index = 5, snapshot term = 1, journal size = 0 01:49:32,607 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 01:49:32,607 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testInstallSnapshot is no longer the leader 01:49:32,607 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from null to Follower 01:49:32,607 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Initializing shard [cars] 01:49:32,607 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testInstallSnapshot is initialized 01:49:32,607 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , registered listener akka://cluster-test/user/shardmanager-testInstallSnapshot 01:49:32,608 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from null to Follower 01:49:32,608 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#1751454855] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:32,611 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:49:32,612 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:32,613 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testInstallSnapshot 01:49:32,614 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: no initial generation override, starting from 0 01:49:32,614 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} 01:49:32,614 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@66121fb5 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@cc0e53a 01:49:32,614 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@58ea9bd2 01:49:32,614 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testInstallSnapshot, 0, 1710164972614, None)) 01:49:32,614 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testInstallSnapshot 01:49:32,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Creating local shard: member-2-shard-cars-testInstallSnapshot 01:49:32,614 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:32,614 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:49:32,614 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164972613,0,0)) 01:49:32,615 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@cc0e53a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@21383c89 01:49:32,615 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} started 01:49:32,615 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@46b8640e 01:49:32,615 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:32,615 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testInstallSnapshot 01:49:32,615 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:49:32,615 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:49:32,615 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: updated peer member-1-shard-cars-testInstallSnapshot on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot on shard member-2-shard-cars-testInstallSnapshot whose actor address is Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot#1757285545] 01:49:32,616 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testInstallSnapshot, persistent : true 01:49:32,616 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testInstallSnapshot with journal batch size 1 01:49:32,616 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/shard-member-2:snapshot-read#226032907] 01:49:32,616 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-notifier#-1361279399 created and ready for shard:member-2-shard-cars-testInstallSnapshot 01:49:32,616 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot#1757285545] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:32,616 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-notifier#-1361279399] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:32,616 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/shard-member-2:snapshot-read#226032907] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:32,617 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:32,617 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:32,617 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 01:49:32,617 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testInstallSnapshot is no longer the leader 01:49:32,617 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testInstallSnapshot set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:49:32,617 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Initializing shard [cars] 01:49:32,617 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testInstallSnapshot , received role change from null to Follower 01:49:32,617 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testInstallSnapshot is initialized 01:49:32,617 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testInstallSnapshot , registered listener akka://cluster-test/user/shardmanager-testInstallSnapshot 01:49:32,617 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-2-shard-cars-testInstallSnapshot from null to Follower 01:49:32,665 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#2132711289] 01:49:32,665 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: findLocalShard : cars 01:49:32,665 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1999229691] 01:49:32,868 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:32,868 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Follower): Received ElectionTimeout - switching to Candidate 01:49:32,868 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testInstallSnapshot] 01:49:32,868 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=2, votedFor=member-1-shard-cars-testInstallSnapshot 01:49:32,868 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 2 01:49:32,869 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Follower) :- Switching from behavior Follower to Candidate, election term: 2 01:49:32,869 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-1-shard-cars-testInstallSnapshot (Follower): Closing 01:49:32,869 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 01:49:32,869 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testInstallSnapshot is no longer the leader 01:49:32,869 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from Follower to Candidate 01:49:32,869 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from Follower to Candidate 01:49:33,98 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 01:49:33,98 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=3, votedFor=member-1-shard-cars-testInstallSnapshot 01:49:33,98 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 3 01:49:33,216 PM [cluster-test-shard-dispatcher-24] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:33,388 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 01:49:33,388 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=4, votedFor=member-1-shard-cars-testInstallSnapshot 01:49:33,388 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 4 01:49:33,520 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:49:33,520 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:33,520 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:33,520 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot 01:49:33,520 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:49:33,520 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:33,520 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:49:33,520 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:49:33,520 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-2-shard-cars-testInstallSnapshot set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:49:33,525 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:33,525 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot 01:49:33,525 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:49:33,525 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:33,525 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:49:33,525 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:49:33,589 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:49:33,698 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 01:49:33,698 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 01:49:33,698 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 5 01:49:33,699 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] to peer member-2-shard-cars-testInstallSnapshot 01:49:33,700 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:33,700 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Cluster state: [] 01:49:33,700 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] with higher term appears reachable, updating term. 01:49:33,700 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Term 5 in "RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1]" message is greater than follower's term 0 - updating term 01:49:33,700 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=null 01:49:33,700 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): In requestVote: RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:33,700 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 01:49:33,700 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:49:33,701 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:49:33,702 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:49:33,702 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Election: Leader has following peers: [member-2-shard-cars-testInstallSnapshot] 01:49:33,702 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: false, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 5 01:49:33,702 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:49:33,702 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: true, hasLeader : true 01:49:33,702 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: became leader with frontend state for [] 01:49:33,702 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from Candidate to Leader 01:49:33,702 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testInstallSnapshot, leaderId=member-1-shard-cars-testInstallSnapshot, leaderPayloadVersion=13] 01:49:33,702 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:33,702 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:33,702 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from Candidate to Leader 01:49:33,702 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot: All Shards are ready - data store testInstallSnapshot is ready 01:49:33,703 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-2-shard-cars-testInstallSnapshot: Closing 01:49:33,703 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: getLeaderAddress leaderId = member-1-shard-cars-testInstallSnapshot peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:49:33,703 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testInstallSnapshot Received follower initial sync status for member-2-shard-cars-testInstallSnapshot status sync done false 01:49:33,703 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testInstallSnapshot, leaderId=member-1-shard-cars-testInstallSnapshot, leaderPayloadVersion=13] 01:49:33,703 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:33,703 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:33,703 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testInstallSnapshot: All Shards are ready - data store testInstallSnapshot is ready 01:49:33,705 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testInstallSnapshot updated: matchIndex: -1, nextIndex: 0 01:49:33,705 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 0, leaderLastIndex: 5, leaderSnapShotIndex: 5 01:49:33,705 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Initiating install snapshot to follower member-2-shard-cars-testInstallSnapshot: follower nextIndex: 0, leader snapshotIndex: 5, leader lastIndex: 5, leader log size: 0 01:49:33,705 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex 5 and lastAppliedTerm 1 instead. 01:49:33,706 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=5, lastAppliedTerm=1, lastIndex=5, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-testInstallSnapshot 01:49:33,706 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: lastSequenceNumber prior to capture: 4 01:49:33,706 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 01:49:33,706 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: CaptureSnapshotReply received by actor 01:49:33,706 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Persisting of snapshot done: Snapshot [lastIndex=5, lastTerm=1, lastAppliedIndex=5, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2b72f71f, electionTerm=5, electionVotedFor=member-1-shard-cars-testInstallSnapshot, ServerConfigPayload=null] 01:49:33,706 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 5 and term: 1 01:49:33,707 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): sendInstallSnapshot 01:49:33,707 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): next snapshot chunk size for follower member-2-shard-cars-testInstallSnapshot: 376 01:49:33,707 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-testInstallSnapshot, member-2-shard-cars-testInstallSnapshot), Chunk: 1/1 01:49:33,707 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: SaveSnapshotSuccess received for snapshot, sequenceNr: 4 01:49:33,707 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Snapshot success - sequence number: 4 01:49:33,707 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: SnapshotComplete received 01:49:33,707 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: handled a journal response 01:49:33,707 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: handled a snapshot response 01:49:33,707 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot Received follower initial sync status for member-2-shard-cars-testInstallSnapshot status sync done true 01:49:33,708 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): handleInstallSnapshot: InstallSnapshot [term=5, leaderId=member-1-shard-cars-testInstallSnapshot, lastIncludedIndex=5, lastIncludedTerm=1, datasize=376, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 01:49:33,708 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 01:49:33,708 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Snapshot installed from leader: member-1-shard-cars-testInstallSnapshot 01:49:33,708 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Applying snapshot on follower: Snapshot [lastIndex=5, lastTerm=1, lastAppliedIndex=5, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@11c59262, electionTerm=5, electionVotedFor=member-1-shard-cars-testInstallSnapshot, ServerConfigPayload=null] 01:49:33,708 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - lastSequenceNumber prior to persisting applied snapshot: 2 01:49:33,708 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: SaveSnapshotSuccess received for snapshot, sequenceNr: 2 01:49:33,708 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Snapshot success - sequence number: 2 01:49:33,709 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Moving last applied index from -1 to 5 01:49:33,709 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 01:49:33,709 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Applying snapshot 01:49:33,709 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendMetadata - member-2-shard-cars-testInstallSnapshot: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 01:49:33,709 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testInstallSnapshot: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-393957163] with name member-2-shard-cars-testInstallSnapshot-DTCL-publisher 01:49:33,709 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-2-shard-cars-testInstallSnapshot: state snapshot applied in 258.3 μs 01:49:33,709 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Done applying snapshot 01:49:33,709 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=5, followerId=member-2-shard-cars-testInstallSnapshot, chunkIndex=1, success=true] 01:49:33,709 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: SnapshotComplete received 01:49:33,709 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-393957163] 01:49:33,709 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testInstallSnapshot: publishChanges: rootPath: / 01:49:33,709 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-393957163] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:33,709 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testInstallSnapshot: Elapsed time for generation of change events for member-2-shard-cars-testInstallSnapshot-DTCL-publisher: 102.0 μs 01:49:33,710 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: handled a journal response 01:49:33,710 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: handled a snapshot response 01:49:33,710 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=5, followerId=member-2-shard-cars-testInstallSnapshot, chunkIndex=1, success=true] 01:49:33,710 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Snapshot successfully installed on follower member-2-shard-cars-testInstallSnapshot (last chunk 1) - matchIndex set to 5, nextIndex set to 6 01:49:33,722 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@47c00a64: creating a new snapshot member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 01:49:33,722 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} 01:49:33,722 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:49:33,722 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:49:33,722 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#2132711289] 01:49:33,722 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:33,722 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, history=0, cookie=0} shard 1 01:49:33,722 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testInstallSnapshot-fe-0-txn-0-1 for transaction member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 01:49:33,722 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1999229691, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4bf68e71] 01:49:33,722 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:49:33,722 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#197601448], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@3c02e6bb 01:49:33,722 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1999229691, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4bf68e71] 01:49:33,722 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testInstallSnapshot: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:49:33,723 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@26d670aa, attempting to connect 01:49:33,723 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: client ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} is not yet known 01:49:33,723 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, nanosAgo=13280, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 01:49:33,723 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:49:33,723 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1999229691], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:49:33,724 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1999229691], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:49:33,724 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@75cd0ab1 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5ca81c7f 01:49:33,724 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:49:33,724 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5ca81c7f 01:49:33,724 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testInstallSnapshot-fe-0-txn-0-1 for transaction member-1-datastore-testInstallSnapshot-fe-0-txn-0-1 01:49:33,724 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} 01:49:33,724 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#197601448], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} 01:49:33,724 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@47c00a64 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1999229691], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:49:33,724 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} 01:49:33,724 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@32237fb8 01:49:33,724 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5b7b71d2 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@47acd526, splicing queue 01:49:33,724 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5b7b71d2 queue spliced 0 messages 01:49:33,724 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1999229691], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 630.7 μs 01:49:33,724 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=0} 01:49:33,726 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 01:49:33,726 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Issuing gracefulStop to shard member-1-shard-cars-testInstallSnapshot 01:49:33,726 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testInstallSnapshot - waiting on 1 shards 01:49:33,726 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onShutDown 01:49:33,726 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Initiating leader transfer 01:49:33,726 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testInstallSnapshot, leaderId=null, leaderPayloadVersion=13] 01:49:33,726 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:33,726 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@25266f75 01:49:33,726 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testInstallSnapshot: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@25266f75 01:49:33,726 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Attempting to transfer leadership 01:49:33,726 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:49:33,726 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 5 01:49:33,727 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@26fb98e1 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@22c418e6, splicing queue 01:49:33,727 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@26fb98e1 queue spliced 0 messages 01:49:33,727 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1999229691], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1999229691], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:49:33,727 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: refreshing backend for shard 1 01:49:33,727 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1999229691], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:49:33,727 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:49:33,727 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:49:33,727 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#2132711289] 01:49:33,727 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:33,727 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testInstallSnapshot, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 01:49:33,728 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testInstallSnapshot] 01:49:33,728 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testInstallSnapshot, leaderId=null, leaderPayloadVersion=13] 01:49:33,728 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:33,728 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:33,729 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testInstallSnapshot, matchIndex: 5, lastIndex: 5, isVoting: true 01:49:33,729 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Follower's log matches - sending ElectionTimeout 01:49:33,729 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 5 01:49:33,729 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Leader transfer complete 01:49:33,940 PM [cluster-test-shard-dispatcher-26] [WARN] TestShard - member-1-shard-cars-testInstallSnapshot (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-2-shard-cars-testInstallSnapshot, logLastIndex=5, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 210, lastApplied : 5, commitIndex : 5 01:49:34,138 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: All shards shutdown - sending PoisonPill to self 01:49:34,138 PM [cluster-test-akka.actor.default-dispatcher-32] [WARN] ShardManager - shard-manager-testInstallSnapshot: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1999229691]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:49:34,139 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-1-shard-cars-testInstallSnapshot 01:49:34,139 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testInstallSnapshot (Leader)_slicer-id-578: Closing 01:49:34,139 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Aborting 0 pending queued transactions 01:49:34,139 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testInstallSnapshot 01:49:34,139 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager$ShardNotInitializedTimeout] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#2132711289] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#2132711289] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#2132711289] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:34,139 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 01:49:34,140 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 01:49:34,140 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: shutting down 01:49:34,140 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#2132711289] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#2132711289] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:34,140 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 01:49:34,140 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 01:49:34,140 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#2132711289] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#2132711289] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:34,140 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#2132711289] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#2132711289] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:34,140 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#197601448] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#197601448] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:34,140 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:34,141 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:34,146 PM [cluster-test-akka.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:34,146 PM [cluster-test-akka.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:34,146 PM [cluster-test-akka.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:34,146 PM [cluster-test-akka.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:34,147 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:34,159 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testInstallSnapshot: Closing 01:49:34,159 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testInstallSnapshot 01:49:34,159 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testInstallSnapshot (Follower): Closing 01:49:34,159 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Aborting 0 pending queued transactions 01:49:34,159 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-testInstallSnapshot 01:49:34,160 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:34,160 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:35,158 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [6205929076642694090] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:35,179 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:35,179 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:35,179 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [800901658721741376] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:35,181 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:35.182] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:35,198 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:35,198 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:35,205 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:35.207] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:35,227 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:35,232 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2170586697081822582] 01:49:35,232 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:35,234 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:35,234 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:35,234 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:35,234 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:35,242 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:35,242 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:35,242 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:35,253 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:35,264 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [7408431056582907582] 01:49:35,264 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:35,266 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:35,266 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:35,273 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:35,274 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:35,281 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:35,286 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [6475592832105040111] 01:49:35,286 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:35,291 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:35,291 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:35,291 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:35,291 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:35,309 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 closeTransaction called 01:49:35,309 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_32#-1264491796)] 01:49:35,309 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 closeTransaction called 01:49:35,309 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_31#1782937953)] 01:49:35,309 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4afcc209 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} 01:49:35,309 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 1 01:49:35,309 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@197fcc88 01:49:35,309 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-804098297], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:35,309 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@78826b00 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} 01:49:35,309 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 1 01:49:35,309 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2017069439], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@17b831d7 01:49:35,309 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1760812235], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:49:35,309 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0} 01:49:35,309 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@75cd0ab1 aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} 01:49:35,309 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:49:35,309 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#197601448], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@1387a7f8 01:49:35,309 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testInstallSnapshot: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} 01:49:35,309 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5ca81c7f aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} 01:49:35,309 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#197601448]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@56853bec 01:49:35,309 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testInstallSnapshot: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1999229691], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 01:49:35,309 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testInstallSnapshot: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1999229691], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:49:35,315 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:35,316 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:35,317 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:35,318 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 01:49:35,323 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-506107171] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:35,324 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:35,325 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:49:35,327 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 01:49:35,327 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:49:35,328 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@b3f72c6 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@439509c3 01:49:35,328 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 01:49:35,328 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2bf1c6b1 01:49:35,328 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:35,328 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:35,328 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:35,328 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1710164975328, None)) 01:49:35,329 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164975327,0,0)) 01:49:35,329 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@439509c3 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2d4eacaa 01:49:35,329 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 01:49:35,330 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:35,331 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:35,332 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 01:49:35,332 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3fa4581d 01:49:35,332 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:35,332 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:35,333 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:35,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:35,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#343094554] 01:49:35,333 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 01:49:35,334 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 01:49:35,334 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#343094554] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:35,335 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-824261525 created and ready for shard:member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:35,335 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-824261525] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:35,335 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:49:35,336 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:35,336 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:35,336 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:49:35,336 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 01:49:35,336 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from null to Follower 01:49:35,336 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Initializing shard [cars] 01:49:35,336 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is initialized 01:49:35,336 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , registered listener akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader 01:49:35,338 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from null to Follower 01:49:35,339 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-1:snapshot-read#486860605] 01:49:35,339 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-1:snapshot-read#486860605] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:35,341 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1931372662] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:35,341 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 01:49:35,341 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:35,342 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:35,342 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:35,342 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 01:49:35,342 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:49:35,342 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@188cec14 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@31235081 01:49:35,342 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7421c8c1 01:49:35,342 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1710164975342, None)) 01:49:35,342 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164975341,0,0)) 01:49:35,343 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3ef0d010 01:49:35,343 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@31235081 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6e938d5d 01:49:35,343 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 01:49:35,343 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:35,343 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:35,343 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:35,343 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:35,343 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1263465734] 01:49:35,344 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 01:49:35,344 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 01:49:35,349 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:49:35,349 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:35,352 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1263465734] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:35,352 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-2:snapshot-read#-1631103903] 01:49:35,352 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#1881129647 created and ready for shard:member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:35,352 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#1881129647] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:35,352 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-2:snapshot-read#-1631103903] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:35,352 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:35,352 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:35,352 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Initializing shard [cars] 01:49:35,352 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader is initialized 01:49:35,352 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:49:35,352 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 01:49:35,353 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from null to Follower 01:49:35,353 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:35,353 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader , registered listener akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader 01:49:35,353 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader from null to Follower 01:49:35,393 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-506107171] 01:49:35,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 01:49:35,393 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#343094554] 01:49:35,622 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:35,622 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Received ElectionTimeout - switching to Candidate 01:49:35,622 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] 01:49:35,622 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:35,623 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 1 01:49:35,623 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:35,623 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Closing 01:49:35,623 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:49:35,623 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 01:49:35,623 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from Follower to Candidate 01:49:35,623 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from Follower to Candidate 01:49:35,922 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 01:49:35,922 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:35,922 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 2 01:49:35,934 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:36,242 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 01:49:36,242 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,242 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 3 01:49:36,253 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:49:36,253 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:36,253 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:36,253 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,253 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,253 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:36,253 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,253 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,253 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,253 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,261 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:36,261 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,261 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,261 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:36,261 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,261 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,261 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,317 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:49:36,542 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 01:49:36,542 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,542 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 4 01:49:36,542 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,542 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,545 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:36,545 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [] 01:49:36,545 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:36,545 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:36,545 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=null 01:49:36,545 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:36,545 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,545 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:49:36,546 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#343094554] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:36,547 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:49:36,547 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:49:36,547 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Election: Leader has following peers: [member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] 01:49:36,547 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:36,547 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:36,547 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:49:36,547 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:49:36,547 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: became leader with frontend state for [] 01:49:36,548 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#343094554] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:36,548 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Closing 01:49:36,548 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: getLeaderAddress leaderId = member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,548 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderPayloadVersion=13] 01:49:36,548 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:36,548 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:36,548 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 01:49:36,548 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader Received follower initial sync status for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader status sync done false 01:49:36,549 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader updated: matchIndex: -1, nextIndex: 0 01:49:36,549 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:36,550 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from Candidate to Leader 01:49:36,550 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderPayloadVersion=13] 01:49:36,550 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:36,550 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:36,550 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from Candidate to Leader 01:49:36,550 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 01:49:36,551 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:36,553 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:36,555 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 01:49:36,558 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-156583400] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:36,559 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:49:36,561 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 01:49:36,561 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 01:49:36,561 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:49:36,561 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,561 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7d51915b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7251b584 01:49:36,561 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@427da694 01:49:36,561 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:36,561 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,561 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:36,561 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,561 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:36,561 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1710164976561, None)) 01:49:36,561 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,562 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164976560,0,0)) 01:49:36,562 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6764e333 01:49:36,562 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7251b584 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@69555ae8 01:49:36,562 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 01:49:36,562 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:36,562 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,562 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,562 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,562 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#-117222813] 01:49:36,562 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,562 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#-117222813] 01:49:36,563 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#-117222813] 01:49:36,563 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 01:49:36,563 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 01:49:36,564 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:49:36,564 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-3:snapshot-read#574207398] 01:49:36,564 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-1934683834 created and ready for shard:member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,564 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:36,564 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 01:49:36,564 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:49:36,565 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:36,565 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:36,569 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:36,569 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:36,569 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:36,569 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:36,570 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:36,587 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-506107171] 01:49:36,588 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 01:49:36,588 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#343094554] 01:49:36,588 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#343094554] 01:49:36,664 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader Received follower initial sync status for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader status sync done true 01:49:37,272 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:49:37,582 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [6475592832105040111] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:37,583 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [6475592832105040111] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:42,372 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 01:49:42,373 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:42,373 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:42,376 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:42,404 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 01:49:42,511 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1931372662] 01:49:42,511 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 01:49:42,511 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1263465734] 01:49:42,511 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#343094554] 01:49:42,511 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:42,511 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:42,511 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 01:49:42,511 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 01:49:42,511 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 01:49:42,511 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 01:49:42,511 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 01:49:42,511 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 01:49:42,511 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 01:49:42,511 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 01:49:42,512 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:42,512 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:42,512 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 01:49:42,512 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 01:49:42,512 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 01:49:42,512 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 01:49:42,512 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): In requestVote: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 01:49:42,512 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 01:49:42,512 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 01:49:42,512 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 01:49:42,512 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#343094554] to Actor[akka://cluster-test/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:42,512 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#343094554] to Actor[akka://cluster-test/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:42,512 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1263465734] to Actor[akka://cluster-test/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:42,512 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1263465734] to Actor[akka://cluster-test/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:42,512 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#343094554] 01:49:42,512 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:49:42,513 PM [main] [WARN] ActorUtils - ShardManager for testSemiReachableCandidateNotDroppingLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.testSemiReachableCandidateNotDroppingLeader(DistributedDataStoreRemotingIntegrationTest.java:1322) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:49:42,513 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:49:42,513 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:49:42,513 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Issuing gracefulStop to shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:42,513 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader - waiting on 1 shards 01:49:42,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onShutDown 01:49:42,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Initiating leader transfer 01:49:42,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@29a2275a 01:49:42,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@29a2275a 01:49:42,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Attempting to transfer leadership 01:49:42,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:42,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:42,514 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 01:49:42,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:42,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:42,520 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader] 01:49:42,520 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 01:49:42,520 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:42,520 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:42,521 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, matchIndex: -1, lastIndex: -1, isVoting: true 01:49:42,521 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Follower's log matches - sending ElectionTimeout 01:49:42,521 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:42,521 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:42,521 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Leader transfer complete 01:49:42,932 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All shards shutdown - sending PoisonPill to self 01:49:42,932 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#343094554]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:49:42,933 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:42,933 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader)_slicer-id-583: Closing 01:49:42,933 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 01:49:42,933 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:49:42,933 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:49:42,933 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:49:42,934 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:49:42,934 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-506107171] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-506107171] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:42,934 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:49:42,934 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:42,934 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:43,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Closing 01:49:43,494 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 01:49:43,894 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [akka.cluster.GossipStatus] from Actor[akka://cluster-test/system/cluster/core/daemon#392230116] to Actor[akka://cluster-test/deadLetters] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:43,894 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [akka.cluster.ddata.Replicator$Internal$Status] from Actor[akka://cluster-test/system/clusterReceptionist/replicator#826293029] to Actor[akka://cluster-test/deadLetters] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:43,894 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [akka.cluster.GossipStatus] from Actor[akka://cluster-test/system/cluster/core/daemon#392230116] to Actor[akka://cluster-test/deadLetters] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:43,954 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2170586697081822582] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:43,955 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:43,955 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:43,957 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:43.959] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:43,965 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:49:43,965 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:43,965 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Closing 01:49:43,965 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 01:49:43,965 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:49:43,966 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:43,966 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:43,968 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:43.969] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:43,980 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:43,986 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [7136350649979071497] 01:49:43,986 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:43,991 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:43,991 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:43,991 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:43,991 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:44,0 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:44,0 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:44,0 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:44,4 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:44,10 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [5905182604597511311] 01:49:44,10 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:44,11 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:44,11 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:44,11 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:44,11 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:44,22 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:44,28 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-1788228016819115354] 01:49:44,28 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:44,32 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:44,32 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:44,38 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:44,39 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:44,41 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:44,41 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:44,42 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadyLocalTransactionForwardedToLeader 01:49:44,51 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 01:49:44,52 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-136612745] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:44,55 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadyLocalTransactionForwardedToLeader 01:49:44,55 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Creating local shard: member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:44,55 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:44,55 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:44,56 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: no initial generation override, starting from 0 01:49:44,56 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 01:49:44,56 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2e1cf814 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@50382307 01:49:44,56 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@59562ac0 01:49:44,56 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, 0, 1710164984056, None)) 01:49:44,56 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164984055,0,0)) 01:49:44,56 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5963e491 01:49:44,56 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@50382307 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7edc7f11 01:49:44,56 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} started 01:49:44,57 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:44,58 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:44,60 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadyLocalTransactionForwardedToLeader 01:49:44,60 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:44,60 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:44,60 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:44,60 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:44,60 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: updated peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497] 01:49:44,61 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:44,61 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, persistent : true 01:49:44,61 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:44,61 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#-221959579 created and ready for shard:member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:44,61 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with journal batch size 1 01:49:44,61 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#-221959579] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:44,61 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-1:snapshot-read#-475185820] 01:49:44,62 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-1:snapshot-read#-475185820] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:44,62 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:44,62 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:44,62 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:49:44,62 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 01:49:44,62 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Initializing shard [cars] 01:49:44,62 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is initialized 01:49:44,63 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from null to Follower 01:49:44,63 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader 01:49:44,65 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from null to Follower 01:49:44,66 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:49:44,70 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:44,70 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:44,70 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 01:49:44,71 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#71552708] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:44,74 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: no initial generation override, starting from 0 01:49:44,74 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 01:49:44,75 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@20aa8002 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@73bfb79e 01:49:44,75 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@36bd378d 01:49:44,75 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, 0, 1710164984075, None)) 01:49:44,75 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testReadyLocalTransactionForwardedToLeader 01:49:44,75 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Creating local shard: member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:44,75 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:44,75 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:44,76 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164984074,0,0)) 01:49:44,76 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@73bfb79e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@37de7ead 01:49:44,76 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} started 01:49:44,76 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@745de841 01:49:44,76 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:44,76 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:44,76 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:44,76 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:44,76 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: updated peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-1828238769] 01:49:44,77 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, persistent : true 01:49:44,77 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-1828238769] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:44,77 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#653665992 created and ready for shard:member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:44,77 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-2:snapshot-read#523626268] 01:49:44,77 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#653665992] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:44,77 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-2:snapshot-read#523626268] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:44,77 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with journal batch size 1 01:49:44,77 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:44,77 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:44,77 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Initializing shard [cars] 01:49:44,77 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader is initialized 01:49:44,77 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:49:44,77 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 01:49:44,78 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:44,78 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from null to Follower 01:49:44,78 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader 01:49:44,78 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader from null to Follower 01:49:44,126 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-136612745] 01:49:44,126 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 01:49:44,127 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497] 01:49:44,308 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:44,309 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Received ElectionTimeout - switching to Candidate 01:49:44,309 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] 01:49:44,309 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:44,309 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 1 01:49:44,309 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:44,309 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Closing 01:49:44,309 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:49:44,309 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 01:49:44,309 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from Follower to Candidate 01:49:44,309 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from Follower to Candidate 01:49:44,588 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:49:44,588 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:44,588 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 2 01:49:44,641 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:44,868 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:49:44,868 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:44,868 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 3 01:49:45,9 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:49:45,9 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:45,10 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:45,10 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,10 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:45,10 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,10 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,52 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:49:45,54 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:45,54 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,54 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,54 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:45,54 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,54 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,138 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:49:45,138 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,138 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 4 01:49:45,138 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,140 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:45,140 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Cluster state: [] 01:49:45,140 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:45,140 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:45,140 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=null 01:49:45,140 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:45,140 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,140 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:49:45,141 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:49:45,141 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:49:45,141 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] 01:49:45,141 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:45,141 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:49:45,141 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:49:45,141 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: became leader with frontend state for [] 01:49:45,141 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from Candidate to Leader 01:49:45,141 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderPayloadVersion=13] 01:49:45,142 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:45,142 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:45,142 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from Candidate to Leader 01:49:45,142 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All Shards are ready - data store testReadyLocalTransactionForwardedToLeader is ready 01:49:45,142 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Closing 01:49:45,142 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,142 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader status sync done false 01:49:45,142 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderPayloadVersion=13] 01:49:45,142 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:45,142 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:45,142 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All Shards are ready - data store testReadyLocalTransactionForwardedToLeader is ready 01:49:45,143 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: -1, nextIndex: 0 01:49:45,143 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:45,233 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#71552708] 01:49:45,233 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 01:49:45,233 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-1828238769] 01:49:45,233 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,233 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#71552708] 01:49:45,233 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 01:49:45,233 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-1828238769] 01:49:45,245 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: handleReadyLocalTransaction for member-1-ShardTransactionTest-fe-0-txn-245-0 01:49:45,245 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,245 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Forwarding ReadyLocalTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader)] 01:49:45,247 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, disabling transaction tracking 01:49:45,247 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:49:45,247 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:49:45,247 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:49:45,247 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:45,247 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,247 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 01:49:45,247 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-245-0 01:49:45,247 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-245-0 of 1 operations, client version 13 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-245-0 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-245-0 ready 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-245-0 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 validated 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-245-0: success 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-245-0 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-245-0 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 candidate ready 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-245-0: canCommit - candidate rootPath: / 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-245-0: canCommit - messages: [] 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-245-0: preCommit - successfulFromPrevious: [] 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 prepared 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-245-0 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-245-0 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 197 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-245-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 submitted to persistence 01:49:45,248 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 01:49:45,248 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:49:45,248 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:49:45,248 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:49:45,248 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:49:45,249 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 1 01:49:45,249 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:45,249 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:49:45,249 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:49:45,249 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:49:45,249 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader status sync done true 01:49:45,249 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 0, nextIndex: 1 01:49:45,249 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:49:45,249 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 0 01:49:45,249 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from -1 to 0 01:49:45,249 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:49:45,249 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 01:49:45,249 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 01:49:45,249 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, nanosAgo=2452600} is already disabled 01:49:45,249 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:45,249 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,250 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 01:49:45,250 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 01:49:45,250 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 01:49:45,250 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:49:45,250 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:49:45,250 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log Simple{size=197} 01:49:45,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:49:45,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 2 01:49:45,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 0 01:49:45,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:45,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:49:45,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 0 01:49:45,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from -1 to 0 01:49:45,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:49:45,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 01:49:45,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 01:49:45,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:49:45,251 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 01:49:45,251 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 1, nextIndex: 2 01:49:45,251 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:49:45,251 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 1 01:49:45,252 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 0 to 1 01:49:45,252 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 01:49:45,252 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-245-0 01:49:45,252 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-245-0: commit - successfulFromPrevious: [] 01:49:45,252 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 changing state from COMMIT_PENDING to COMMITTED 01:49:45,252 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/system/testProbe-764#506472325] 01:49:45,252 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-562743163] with name member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher 01:49:45,252 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-562743163] 01:49:45,252 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 01:49:45,252 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 01:49:45,252 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:45,252 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 01:49:45,252 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 128.5 μs 01:49:45,252 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-562743163] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:45,253 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@63321928: creating a new snapshot member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 01:49:45,253 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 1 01:49:45,253 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:49:45,253 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 1 01:49:45,253 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 0 to 1 01:49:45,253 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 01:49:45,253 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} 01:49:45,253 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:49:45,253 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:49:45,253 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-136612745] 01:49:45,253 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, history=0, cookie=0} shard 1 01:49:45,253 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-245-0 01:49:45,253 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1 for transaction member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 01:49:45,253 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:49:45,253 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#259906372], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@71c0fe45 01:49:45,254 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:49:45,254 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:45,254 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@518a7f3f] 01:49:45,254 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@518a7f3f] 01:49:45,254 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-866891379] with name member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher 01:49:45,254 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-866891379] 01:49:45,254 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 01:49:45,254 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 01:49:45,254 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-866891379] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:45,254 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 01:49:45,254 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 90.51 μs 01:49:45,254 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@692c803b, attempting to connect 01:49:45,254 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: client ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} is not yet known 01:49:45,255 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, nanosAgo=9290, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 01:49:45,255 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:49:45,257 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:49:45,257 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:49:45,257 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@382b320e instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3d0013a3 01:49:45,257 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:49:45,257 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3d0013a3 01:49:45,257 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1 for transaction member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1 01:49:45,257 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 01:49:45,257 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#259906372], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 01:49:45,257 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@63321928 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:49:45,257 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} 01:49:45,257 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@135d3331 01:49:45,257 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2ddea476 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@10fe3765, splicing queue 01:49:45,257 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2ddea476 queue spliced 0 messages 01:49:45,258 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 677.3 μs 01:49:45,258 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0} 01:49:45,258 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: handleReadyLocalTransaction for member-1-ShardTransactionTest-fe-0-txn-246-0 01:49:45,258 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,258 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Forwarding ReadyLocalTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader)] 01:49:45,260 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-246-0 01:49:45,260 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-246-0 of 1 operations, client version 13 01:49:45,260 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-246-0 01:49:45,260 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-246-0 ready 01:49:45,261 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0 canCommit 01:49:45,261 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497)] resolved 01:49:45,261 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: successfully resolved all cohort actors 01:49:45,261 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0 finishCanCommit 01:49:45,261 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: sending CanCommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-246-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497)] 01:49:45,261 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497)] 01:49:45,261 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Can committing transaction member-1-ShardTransactionTest-fe-0-txn-246-0 01:49:45,261 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-246-0 01:49:45,261 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 validated 01:49:45,261 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:45,261 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-246-0: success 01:49:45,261 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:49:45,261 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: canCommit returning result: true 01:49:45,261 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0 commit 01:49:45,261 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0 finish commit 01:49:45,261 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: Sending CommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-246-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497)] 01:49:45,261 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497)] 01:49:45,261 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-246-0 01:49:45,261 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-246-0 01:49:45,261 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 candidate ready 01:49:45,261 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-246-0: canCommit - candidate rootPath: / 01:49:45,261 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-246-0: canCommit - messages: [] 01:49:45,261 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-246-0: preCommit - successfulFromPrevious: [] 01:49:45,261 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 prepared 01:49:45,261 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:45,261 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-246-0 01:49:45,261 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-246-0 01:49:45,262 PM [cluster-test-shard-dispatcher-24] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 208 01:49:45,262 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:49:45,262 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:49:45,262 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-246-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:45,262 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:49:45,262 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,262 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 01:49:45,262 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 submitted to persistence 01:49:45,262 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:49:45,263 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 01:49:45,263 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:49:45,263 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:49:45,263 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log Simple{size=208} 01:49:45,263 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:49:45,263 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 2 01:49:45,263 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:45,263 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 2, nextIndex: 3 01:49:45,263 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:49:45,263 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 2 01:49:45,263 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 1 to 2 01:49:45,263 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 01:49:45,263 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-246-0 01:49:45,263 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-246-0: commit - successfulFromPrevious: [] 01:49:45,263 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 changing state from COMMIT_PENDING to COMMITTED 01:49:45,263 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testReadyLocalTransactionForwardedToLeader_member-1-shard-cars-testReadyLocalTransactionForwardedToLeader%23-518262497$a] 01:49:45,264 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: commit succeeded 01:49:45,264 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 01:49:45,264 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:49:45,264 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 01:49:45,264 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 01:49:45,264 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@63321928: creating a new snapshot member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0 01:49:45,264 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 71.01 μs 01:49:45,264 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-1 for transaction member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0 01:49:45,264 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 01:49:45,264 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Issuing gracefulStop to shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,264 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader - waiting on 1 shards 01:49:45,264 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onShutDown 01:49:45,264 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Initiating leader transfer 01:49:45,264 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@51e3d7c4 01:49:45,265 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@51e3d7c4 01:49:45,265 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Attempting to transfer leadership 01:49:45,265 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:49:45,265 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 01:49:45,265 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:45,265 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:49:45,265 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3da99434 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@71103abd, splicing queue 01:49:45,265 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3da99434 queue spliced 0 messages 01:49:45,265 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:49:45,265 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: refreshing backend for shard 1 01:49:45,265 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:49:45,265 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:49:45,265 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:49:45,265 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-136612745] 01:49:45,265 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:45,265 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 01:49:45,268 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:49:45,268 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 2 01:49:45,268 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:49:45,268 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 2 01:49:45,268 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 1 to 2 01:49:45,268 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 01:49:45,268 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-246-0 01:49:45,268 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 01:49:45,268 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 01:49:45,269 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader] 01:49:45,269 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 01:49:45,269 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 67.85 μs 01:49:45,269 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 01:49:45,269 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:45,269 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:45,269 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, matchIndex: 2, lastIndex: 2, isVoting: true 01:49:45,269 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Follower's log matches - sending ElectionTimeout 01:49:45,269 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:49:45,269 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Leader transfer complete 01:49:45,679 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received ShardNotInitializedTimeout message for shard cars 01:49:45,679 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 01:49:45,679 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Returning NoShardLeaderException for shard cars 01:49:45,679 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:49:45,679 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:49:45,679 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,679 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader)_slicer-id-590: Closing 01:49:45,680 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Aborting 0 pending queued transactions 01:49:45,680 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 01:49:45,680 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 01:49:45,680 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 01:49:45,680 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:49:45,680 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 01:49:45,680 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:49:45,681 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: shutting down 01:49:45,681 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 01:49:45,681 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-136612745] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-136612745] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:45,681 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-136612745] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-136612745] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:45,681 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-136612745] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-136612745] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:45,682 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:45,682 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:45,688 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:45,688 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:45,688 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:45,689 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:45,691 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:45,700 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 01:49:45,701 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:49:45,701 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Closing 01:49:45,701 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Aborting 0 pending queued transactions 01:49:45,701 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 01:49:45,702 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:45,702 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:46,700 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [7136350649979071497] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:46,721 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [5905182604597511311] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:46,723 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:46,723 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:46,725 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:46.726] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:46,732 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:46,732 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:46,733 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:46.734] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:46,748 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:46,754 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-5767626558629819300] 01:49:46,754 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:46,756 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:46,756 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:46,756 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:46,756 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:46,768 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:46,768 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:46,768 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:46,769 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:46,780 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-2131775139635998893] 01:49:46,780 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:46,786 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:46,786 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:46,789 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:46,789 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:46,798 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:46,812 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [1782862732752470690] 01:49:46,813 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:46,814 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:46,814 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:46,822 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:46,822 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:46,827 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:46,829 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:46,830 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithSingleShard 01:49:46,830 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:46,833 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-1173940702] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:46,833 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:46,838 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithSingleShard 01:49:46,848 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: no initial generation override, starting from 0 01:49:46,848 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:49:46,848 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6855fc24 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@22f4f777 01:49:46,848 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@56fcf5ea 01:49:46,848 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithSingleShard 01:49:46,848 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Creating local shard: member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:49:46,849 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:46,849 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:49:46,850 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, 0, 1710164986850, None)) 01:49:46,850 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164986849,0,0)) 01:49:46,850 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@59f0e37d 01:49:46,850 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@22f4f777 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3433396 01:49:46,850 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} started 01:49:46,851 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:46,852 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:46,852 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:49:46,852 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:46,854 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithSingleShard 01:49:46,857 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:46,859 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:49:46,859 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:49:46,859 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:49:46,859 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testReadWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard on shard member-1-shard-cars-testReadWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-868543854] 01:49:46,859 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testReadWriteTransactionWithSingleShard, persistent : true 01:49:46,859 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-868543854] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:46,860 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-1102662324 created and ready for shard:member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:49:46,860 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-1102662324] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:46,860 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-1:snapshot-read#1339735387] 01:49:46,860 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadWriteTransactionWithSingleShard with journal batch size 1 01:49:46,860 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-1:snapshot-read#1339735387] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:46,861 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:46,861 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:46,861 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:49:46,861 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 01:49:46,861 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Initializing shard [cars] 01:49:46,861 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is initialized 01:49:46,864 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithSingleShard 01:49:46,864 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from null to Follower 01:49:46,864 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard 01:49:46,864 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from null to Follower 01:49:46,864 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#352562293] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:46,869 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithSingleShard 01:49:46,869 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: no initial generation override, starting from 0 01:49:46,869 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:49:46,869 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:49:46,869 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@16ccdd76 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@faff4d6 01:49:46,869 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@67564404 01:49:46,869 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:46,869 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:49:46,869 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, 0, 1710164986869, None)) 01:49:46,869 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164986868,0,0)) 01:49:46,870 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4f46ef62 01:49:46,870 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@faff4d6 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@461b73f8 01:49:46,870 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} started 01:49:46,870 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:46,870 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:49:46,871 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:49:46,871 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:49:46,871 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testReadWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard on shard member-2-shard-cars-testReadWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard#-1287882844] 01:49:46,871 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testReadWriteTransactionWithSingleShard, persistent : true 01:49:46,872 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadWriteTransactionWithSingleShard with journal batch size 1 01:49:46,872 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard#-1287882844] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:46,872 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-1809096528] 01:49:46,872 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-notifier#802457430 created and ready for shard:member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:49:46,872 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-1809096528] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:46,872 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-notifier#802457430] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:46,872 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:46,872 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:46,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Initializing shard [cars] 01:49:46,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard is initialized 01:49:46,872 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:49:46,872 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 01:49:46,872 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:49:46,872 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithSingleShard , received role change from null to Follower 01:49:46,872 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard 01:49:46,873 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testReadWriteTransactionWithSingleShard from null to Follower 01:49:46,920 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-1173940702] 01:49:46,920 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: findLocalShard : cars 01:49:46,920 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-868543854] 01:49:47,136 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:47,136 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 01:49:47,136 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadWriteTransactionWithSingleShard] 01:49:47,136 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:49:47,136 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 1 01:49:47,136 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:47,136 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Closing 01:49:47,136 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:49:47,136 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 01:49:47,136 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from Follower to Candidate 01:49:47,136 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from Follower to Candidate 01:49:47,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:49:47,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:49:47,385 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 2 01:49:47,468 PM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:47,705 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:49:47,705 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:49:47,705 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 3 01:49:47,777 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:49:47,777 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:47,777 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:47,777 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard 01:49:47,777 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:49:47,777 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:47,777 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:49:47,777 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:49:47,777 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:49:47,785 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:47,785 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard 01:49:47,785 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:49:47,785 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:47,785 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:49:47,785 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:49:47,835 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:49:47,955 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:49:47,955 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:49:47,956 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 4 01:49:47,956 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:49:47,957 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:47,957 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Cluster state: [] 01:49:47,957 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:47,957 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:47,957 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 01:49:47,957 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:47,957 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:49:47,957 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:49:47,958 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:49:47,958 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:49:47,958 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadWriteTransactionWithSingleShard] 01:49:47,958 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:47,958 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:49:47,958 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:49:47,958 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: became leader with frontend state for [] 01:49:47,959 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Closing 01:49:47,959 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testReadWriteTransactionWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:49:47,959 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithSingleShard status sync done false 01:49:47,959 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:49:47,959 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:47,959 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:47,959 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All Shards are ready - data store testReadWriteTransactionWithSingleShard is ready 01:49:47,960 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: -1, nextIndex: 0 01:49:47,960 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:47,963 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from Candidate to Leader 01:49:47,963 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:49:47,963 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:47,963 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:47,963 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from Candidate to Leader 01:49:47,963 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All Shards are ready - data store testReadWriteTransactionWithSingleShard is ready 01:49:47,976 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6d13fb89: creating a new transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:49:47,977 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} 01:49:47,977 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:49:47,977 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:49:47,977 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#352562293] 01:49:47,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:47,977 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} shard 1 01:49:47,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard, primaryVersion=13] 01:49:47,977 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:49:47,977 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:49:47,977 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard, primaryVersion=13] 01:49:47,977 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 1 01:49:47,977 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=3} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@73124497 01:49:47,977 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:49:47,977 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@22b6ae8e, attempting to connect 01:49:47,977 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1666168648], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@471cd42e 01:49:47,978 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} is not yet known 01:49:47,978 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, nanosAgo=12270, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:49:47,979 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:49:47,979 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-868543854], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:49:47,980 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-868543854], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:47,980 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@649fdca8 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2b225d8f 01:49:47,980 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:49:47,980 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2b225d8f 01:49:47,980 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:47,980 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 01:49:47,980 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=3} to successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 01:49:47,980 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:49:47,980 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1666168648], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 01:49:47,980 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} enqueing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=3} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2250/0x00000008016bde88@169bfb7d 01:49:47,981 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-868543854], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:49:47,981 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2f6c0204 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=3}, enqueuedTicks=734312481433} 01:49:47,981 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-593: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=3}} 01:49:47,981 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-593: Message size: 467 bytes, total slices to send: 1 01:49:47,981 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-593: Message does not need to be sliced - sending original message 01:49:47,981 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 1 01:49:47,981 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1666168648], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1248/0x0000000801301d10@6c3efc08 01:49:47,981 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2f6c0204 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1666168648], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=734312481433} 01:49:47,981 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6d13fb89 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-868543854], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:47,981 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 01:49:47,981 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@370135dd 01:49:47,981 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2413e5e2 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3f2c5553, splicing queue 01:49:47,981 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2413e5e2 queue spliced 0 messages 01:49:47,982 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-868543854], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.898 ms 01:49:47,984 PM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithSingleShard: allocating new open transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:47,984 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-592: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 01:49:47,985 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-592: Message size: 372 bytes, total slices to send: 1 01:49:47,985 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=3} with Envelope{sessionId=0, txSequence=0, message=ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0}} 01:49:47,985 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-592: Message does not need to be sliced - sending original message 01:49:47,985 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=3}, enqueuedTicks=734312481433, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0}} 01:49:47,985 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=3} completed with ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0} 01:49:47,985 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1666168648], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 01:49:47,985 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1666168648], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=734312481433, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 01:49:47,985 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1} 01:49:47,985 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:49:47,985 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 01:49:47,986 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@c3ba3af 01:49:47,986 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} 01:49:47,986 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 2 01:49:47,986 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} sending request ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1666168648], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1240/0x00000008013035b8@6dec6843 01:49:47,986 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} 01:49:47,986 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@4c706c74 01:49:47,986 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-868543854], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:47,986 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2f6c0204 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=1}, enqueuedTicks=734320952532} 01:49:47,986 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-593: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=1}} 01:49:47,986 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-593: Message size: 457 bytes, total slices to send: 1 01:49:47,986 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-593: Message does not need to be sliced - sending original message 01:49:47,986 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 3 01:49:47,986 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1666168648], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1237/0x0000000801302f20@25d95134 01:49:47,986 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2f6c0204 transmitting entry ConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1666168648], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]}, enqueuedTicks=734321460250} 01:49:47,991 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=1} with Envelope{sessionId=0, txSequence=2, message=ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2}} 01:49:47,991 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=1}, enqueuedTicks=734320952532, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2}} 01:49:47,991 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=1} completed with ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2} 01:49:47,992 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1666168648], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} with Envelope{sessionId=0, txSequence=3, message=ExistsTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, exists=true}} 01:49:47,992 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1666168648], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]}, enqueuedTicks=734321460250, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=ExistsTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, exists=true}} 01:49:47,992 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Exists request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] completed with ExistsTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, exists=true} 01:49:47,993 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6d13fb89 readied transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:49:47,993 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED} allocated sequence 4 01:49:47,993 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@1722656 01:49:47,993 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-868543854], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:47,993 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2f6c0204 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=0, protocol=SIMPLE}, enqueuedTicks=734328238492} 01:49:47,993 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-593: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=0, protocol=SIMPLE}} 01:49:47,993 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-593: Message size: 289 bytes, total slices to send: 1 01:49:47,993 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-593: Message does not need to be sliced - sending original message 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: readying transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 ready 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithSingleShard: transitioned member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 to ready 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct canCommit 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 validated 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct preCommit 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 candidate ready 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1: canCommit - candidate rootPath: / 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1: canCommit - messages: [] 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 prepared 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct commit 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 250 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 01:49:47,994 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:47,995 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:47,995 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:49:47,995 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}]]] 01:49:47,995 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 submitted to persistence 01:49:47,995 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 01:49:47,996 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}]]] 01:49:47,996 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 01:49:47,996 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:49:47,996 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=250} 01:49:47,996 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 01:49:47,996 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Log size is now 1 01:49:47,996 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:47,996 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 01:49:47,996 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithSingleShard status sync done true 01:49:47,997 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: 0, nextIndex: 1 01:49:47,997 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:49:47,997 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Setting last applied to 0 01:49:47,997 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 01:49:47,997 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 0 data Simple{size=250} 01:49:47,997 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:47,997 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@722eb245}, purged=MutableUnsignedLongSet{size=0}} 01:49:47,997 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:49:47,997 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:47,997 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1: commit - successfulFromPrevious: [] 01:49:47,997 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:49:47,997 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1065504150] with name member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher 01:49:47,998 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1065504150] 01:49:47,998 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:49:47,998 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:47,998 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1065504150] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:47,998 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 01:49:47,998 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher: 118.7 μs 01:49:47,999 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4}} 01:49:47,999 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=0, protocol=SIMPLE}, enqueuedTicks=734328238492, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4}} 01:49:47,999 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED} directCommit completed 01:49:47,999 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED}: initiating purge 01:49:47,999 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} allocated sequence 5 01:49:47,999 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6d13fb89: creating a new snapshot member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 01:49:47,999 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#1666168648]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@24693002 01:49:47,999 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-868543854], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:47,999 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2f6c0204 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#1666168648]}, enqueuedTicks=734333890805} 01:49:47,999 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1 for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 01:49:47,999 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:49:47,999 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1666168648], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@6ac92c03 01:49:47,999 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2f6c0204 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1666168648], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=734334136119} 01:49:47,999 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Commit index set to 0 01:49:47,999 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:49:47,999 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Setting last applied to 0 01:49:48,0 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 01:49:48,0 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 0 data Simple{size=250} 01:49:48,0 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:48,0 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@21e743d6}, purged=MutableUnsignedLongSet{size=0}} 01:49:48,0 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:49:48,0 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:48,0 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: purging transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:48,0 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#752399268] with name member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher 01:49:48,0 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 01:49:48,0 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 01:49:48,0 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:49:48,0 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 01:49:48,0 PM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithSingleShard: allocating new open snapshot member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1 01:49:48,0 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-592: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 01:49:48,1 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-592: Message size: 395 bytes, total slices to send: 1 01:49:48,1 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-592: Message does not need to be sliced - sending original message 01:49:48,1 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 01:49:48,1 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#752399268] 01:49:48,1 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:49:48,1 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#752399268] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:48,1 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher: 82.77 μs 01:49:48,1 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:49:48,1 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}]]] 01:49:48,2 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1666168648], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 01:49:48,2 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1666168648], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=734334136119, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 01:49:48,2 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0} 01:49:48,2 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:49:48,3 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:49:48,3 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithSingleShard - waiting on 1 shards 01:49:48,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onShutDown 01:49:48,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Initiating leader transfer 01:49:48,3 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:49:48,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}]]] 01:49:48,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6e325a20 01:49:48,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 01:49:48,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:49:48,3 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:48,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69} 01:49:48,3 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:48,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 01:49:48,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6e325a20 01:49:48,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Log size is now 2 01:49:48,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Attempting to transfer leadership 01:49:48,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:48,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:48,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:49:48,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 01:49:48,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: 1, nextIndex: 2 01:49:48,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:49:48,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Setting last applied to 1 01:49:48,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 01:49:48,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard] 01:49:48,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69} 01:49:48,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: replication of member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@3be1310c 01:49:48,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithSingleShard: finished purging transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:48,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:49:48,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:48,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 01:49:48,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:48,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadWriteTransactionWithSingleShard, matchIndex: 1, lastIndex: 1, isVoting: true 01:49:48,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 01:49:48,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:48,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Leader transfer complete 01:49:48,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 01:49:48,4 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:49:48,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:48,4 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:49:48,4 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2f6c0204 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@72e23eb0, splicing queue 01:49:48,5 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-868543854], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:49:48,5 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2f6c0204 queue spliced 1 messages 01:49:48,5 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-868543854], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-868543854], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:48,5 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: refreshing backend for shard 1 01:49:48,5 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-868543854], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:49:48,5 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:49:48,5 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:49:48,5 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#352562293] 01:49:48,5 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:48,5 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:49:48,6 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Commit index set to 1 01:49:48,6 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:49:48,6 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Setting last applied to 1 01:49:48,6 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 01:49:48,6 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69} 01:49:48,6 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:49:48,6 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:48,7 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5}} not found in inflight queue, checking pending queue 01:49:48,7 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 01:49:48,7 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#1666168648]} with Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5}} 01:49:48,7 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#1666168648]}, enqueuedTicks=734333890805, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5}} 01:49:48,7 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE}: purge completed 01:49:48,7 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 01:49:48,7 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2b225d8f purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} 01:49:48,415 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 01:49:48,415 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-868543854]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:49:48,415 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:49:48,416 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader)_slicer-id-595: Closing 01:49:48,416 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:49:48,416 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithSingleShard 01:49:48,416 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 01:49:48,416 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:49:48,416 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: shutting down 01:49:48,416 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 01:49:48,416 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 01:49:48,416 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-1173940702] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-1173940702] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:48,417 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#-254178482] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-254178482] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:48,417 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#-254178482] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-254178482] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:48,417 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:48,417 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:48,421 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:48,421 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:48,422 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:48,422 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:48,427 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 01:49:48,427 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:49:48,427 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Closing 01:49:48,427 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:49:48,427 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithSingleShard 01:49:48,428 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:48,428 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:49,435 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-5767626558629819300] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:49,445 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-2131775139635998893] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:49,450 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:49,450 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:49,458 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:49.460] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:49,470 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:49,470 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:49,473 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:49.475] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:49,493 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:49:49,499 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-2421982606466426472] 01:49:49,499 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:49,501 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:49,501 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:49,501 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:49,501 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:49,512 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:49,512 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:49,512 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:49,519 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:49,526 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [5990704085422017429] 01:49:49,526 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:49,527 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:49,527 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:49,546 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:49,546 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:49,559 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:49,567 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [7205324237586680055] 01:49:49,567 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:49,573 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:49,573 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:49,574 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:49,574 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:49,584 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@382b320e aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} 01:49:49,584 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:49:49,584 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#259906372], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@67190f49 01:49:49,584 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} 01:49:49,584 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3d0013a3 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 01:49:49,584 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#259906372]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@490fdc35 01:49:49,584 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 01:49:49,584 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:49:49,584 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3d0013a3 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-1, state=OPEN} 01:49:49,584 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#259906372]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@36aac4a3 01:49:49,584 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 01:49:49,584 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-518262497], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:49:49,584 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2b225d8f aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} 01:49:49,584 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} allocated sequence 1 01:49:49,584 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1666168648], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@4a9e7c38 01:49:49,584 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-868543854], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:49,584 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-868543854], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:49,584 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0} 01:49:49,587 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:49,588 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:49,589 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSnapshotOnRootOverwrite 01:49:49,592 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:49,593 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-testSnapshotOnRootOverwrite 01:49:49,594 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1195081643] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,595 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:49,601 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: no initial generation override, starting from 0 01:49:49,602 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:49:49,602 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@749f6a5f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@38eaf0f1 01:49:49,602 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Recovery complete : shard-manager-testSnapshotOnRootOverwrite 01:49:49,602 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-1-shard-cars-testSnapshotOnRootOverwrite 01:49:49,602 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-1-shard-default-testSnapshotOnRootOverwrite 01:49:49,602 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:49,602 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:49:49,602 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:49:49,603 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2a0d2537 01:49:49,603 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSnapshotOnRootOverwrite, 0, 1710164989602, None)) 01:49:49,606 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164989601,0,0)) 01:49:49,606 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@38eaf0f1 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5624742b 01:49:49,606 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} started 01:49:49,607 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:49,608 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:49,610 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSnapshotOnRootOverwrite 01:49:49,610 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6838573a 01:49:49,610 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:49,610 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSnapshotOnRootOverwrite 01:49:49,611 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:49:49,611 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:49:49,611 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-cars-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite on shard member-1-shard-cars-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1438078722] 01:49:49,611 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-default-testSnapshotOnRootOverwrite 01:49:49,611 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:49:49,611 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:49:49,611 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:49:49,611 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-default-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite on shard member-1-shard-default-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-304630517] 01:49:49,611 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Shard created : member-1-shard-cars-testSnapshotOnRootOverwrite, persistent : true 01:49:49,611 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1438078722] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,612 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-1-shard-default-testSnapshotOnRootOverwrite, persistent : true 01:49:49,612 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-304630517] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,613 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-1-shard-default-testSnapshotOnRootOverwrite with journal batch size 1 01:49:49,613 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-notifier#85479435 created and ready for shard:member-1-shard-cars-testSnapshotOnRootOverwrite 01:49:49,613 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-notifier#85479435] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,614 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-635337645] 01:49:49,615 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:49,615 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:49,615 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:49:49,615 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-default-testSnapshotOnRootOverwrite is no longer the leader 01:49:49,615 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [default] 01:49:49,615 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-1-shard-default-testSnapshotOnRootOverwrite is initialized 01:49:49,617 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSnapshotOnRootOverwrite with journal batch size 1 01:49:49,617 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:49,617 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:49,617 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [cars] 01:49:49,617 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSnapshotOnRootOverwrite is initialized 01:49:49,617 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:49:49,617 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 01:49:49,617 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from null to Follower 01:49:49,617 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:49,618 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 01:49:49,618 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1800912492] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,618 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSnapshotOnRootOverwrite 01:49:49,618 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from null to Follower 01:49:49,619 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-635337645] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,619 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-notifier#1903449817 created and ready for shard:member-1-shard-default-testSnapshotOnRootOverwrite 01:49:49,619 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from null to Follower 01:49:49,619 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 01:49:49,619 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-notifier#1903449817] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,620 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#1533115305] 01:49:49,620 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#1533115305] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,621 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSnapshotOnRootOverwrite 01:49:49,621 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: no initial generation override, starting from 0 01:49:49,621 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:49:49,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-2-shard-cars-testSnapshotOnRootOverwrite 01:49:49,621 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@51e3cb33 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6a7bf011 01:49:49,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-2-shard-default-testSnapshotOnRootOverwrite 01:49:49,622 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:49,622 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:49:49,622 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:49:49,622 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6e447986 01:49:49,622 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSnapshotOnRootOverwrite, 0, 1710164989621, None)) 01:49:49,622 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164989620,0,0)) 01:49:49,623 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6a7bf011 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@36c5fe9e 01:49:49,623 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@36b8402 01:49:49,623 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} started 01:49:49,623 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:49,623 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSnapshotOnRootOverwrite 01:49:49,623 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:49:49,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:49:49,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-cars-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite on shard member-2-shard-cars-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-1649343470] 01:49:49,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-default-testSnapshotOnRootOverwrite 01:49:49,624 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:49:49,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:49:49,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-default-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite on shard member-2-shard-default-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite#1094278254] 01:49:49,624 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-default-testSnapshotOnRootOverwrite, persistent : true 01:49:49,624 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testSnapshotOnRootOverwrite, persistent : true 01:49:49,624 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-default-testSnapshotOnRootOverwrite with journal batch size 1 01:49:49,624 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-notifier#2052885694 created and ready for shard:member-2-shard-default-testSnapshotOnRootOverwrite 01:49:49,624 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSnapshotOnRootOverwrite with journal batch size 1 01:49:49,624 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#31262547] 01:49:49,625 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#-765194683] 01:49:49,625 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-1649343470] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,625 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite#1094278254] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,625 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-notifier#2052885694] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,625 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#31262547] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,625 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#-765194683] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,625 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:49,625 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:49,625 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [default] 01:49:49,625 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-default-testSnapshotOnRootOverwrite is initialized 01:49:49,625 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:49:49,625 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-2-shard-default-testSnapshotOnRootOverwrite is no longer the leader 01:49:49,625 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-default-testSnapshotOnRootOverwrite , received role change from null to Follower 01:49:49,625 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-default-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:49:49,625 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-default-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 01:49:49,626 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:49,626 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:49,626 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:49:49,626 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 01:49:49,626 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [cars] 01:49:49,626 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSnapshotOnRootOverwrite is initialized 01:49:49,626 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:49:49,626 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-2-shard-default-testSnapshotOnRootOverwrite from null to Follower 01:49:49,627 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-notifier#1297094669 created and ready for shard:member-2-shard-cars-testSnapshotOnRootOverwrite 01:49:49,627 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSnapshotOnRootOverwrite , received role change from null to Follower 01:49:49,627 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 01:49:49,627 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-notifier#1297094669] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,627 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-2-shard-cars-testSnapshotOnRootOverwrite from null to Follower 01:49:49,628 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from null to Follower 01:49:49,673 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1195081643] 01:49:49,673 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 01:49:49,673 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1438078722] 01:49:49,849 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:49,849 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Received ElectionTimeout - switching to Candidate 01:49:49,849 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSnapshotOnRootOverwrite] 01:49:49,849 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=1, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 01:49:49,849 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 1 01:49:49,849 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:49,849 PM [cluster-test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Closing 01:49:49,849 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:49:49,849 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 01:49:49,850 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from Follower to Candidate 01:49:49,850 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from Follower to Candidate 01:49:49,868 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:49,868 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Received ElectionTimeout - switching to Candidate 01:49:49,868 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Election: Candidate has following voting peers: [member-2-shard-default-testSnapshotOnRootOverwrite] 01:49:49,868 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=1, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 01:49:49,868 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 1 01:49:49,868 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:49,868 PM [cluster-test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Closing 01:49:49,868 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:49:49,868 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-default-testSnapshotOnRootOverwrite is no longer the leader 01:49:49,868 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from Follower to Candidate 01:49:49,869 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from Follower to Candidate 01:49:50,109 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:49:50,109 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=2, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 01:49:50,109 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 2 01:49:50,118 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:49:50,118 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=2, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 01:49:50,118 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 2 01:49:50,193 PM [cluster-test-shard-dispatcher-32] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:50,233 PM [cluster-test-shard-dispatcher-32] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:50,389 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:49:50,389 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 01:49:50,389 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 3 01:49:50,399 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:49:50,399 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 01:49:50,399 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 3 01:49:50,520 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:49:50,520 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:50,521 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:50,521 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite 01:49:50,521 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:49:50,521 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite 01:49:50,521 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:49:50,521 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:50,521 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:49:50,521 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:49:50,521 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:49:50,521 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:49:50,521 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:49:50,521 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Peer address for peer member-2-shard-default-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:49:50,532 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:50,532 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite 01:49:50,532 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:49:50,532 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite 01:49:50,532 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:49:50,532 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:50,532 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:49:50,532 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:49:50,532 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:49:50,532 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:49:50,582 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:49:50,678 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:49:50,678 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:49:50,678 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 01:49:50,678 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 4 01:49:50,678 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 01:49:50,678 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 4 01:49:50,678 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-default-testSnapshotOnRootOverwrite 01:49:50,679 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSnapshotOnRootOverwrite 01:49:50,680 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:50,680 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Cluster state: [] 01:49:50,680 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:50,680 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:50,680 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=null 01:49:50,680 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:50,680 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 01:49:50,680 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:49:50,680 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:50,680 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Cluster state: [] 01:49:50,680 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:50,680 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:50,680 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=null 01:49:50,680 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:50,680 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 01:49:50,680 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:49:50,682 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:49:50,682 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:49:50,682 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Election: Leader has following peers: [member-2-shard-default-testSnapshotOnRootOverwrite] 01:49:50,682 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:49:50,682 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:49:50,682 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:50,682 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Election: Leader has following peers: [member-2-shard-cars-testSnapshotOnRootOverwrite] 01:49:50,683 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:49:50,683 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:50,683 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: true, hasLeader : true 01:49:50,683 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: became leader with frontend state for [] 01:49:50,683 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:49:50,683 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: true, hasLeader : true 01:49:50,683 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from Candidate to Leader 01:49:50,683 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: became leader with frontend state for [] 01:49:50,683 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 01:49:50,683 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from Candidate to Leader 01:49:50,683 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 01:49:50,683 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from Candidate to Leader 01:49:50,683 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 01:49:50,683 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 01:49:50,683 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:50,683 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:50,683 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from Candidate to Leader 01:49:50,683 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All Shards are ready - data store testSnapshotOnRootOverwrite is ready 01:49:50,684 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-default-testSnapshotOnRootOverwrite: Closing 01:49:50,684 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: getLeaderAddress leaderId = member-1-shard-default-testSnapshotOnRootOverwrite peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:49:50,684 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-testSnapshotOnRootOverwrite, leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 01:49:50,684 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 01:49:50,684 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testSnapshotOnRootOverwrite: Closing 01:49:50,684 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-default-testSnapshotOnRootOverwrite status sync done false 01:49:50,684 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: getLeaderAddress leaderId = member-1-shard-cars-testSnapshotOnRootOverwrite peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:49:50,684 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSnapshotOnRootOverwrite, leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 01:49:50,684 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 01:49:50,684 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:50,684 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:50,684 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All Shards are ready - data store testSnapshotOnRootOverwrite is ready 01:49:50,684 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-cars-testSnapshotOnRootOverwrite status sync done false 01:49:50,685 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: -1, nextIndex: 0 01:49:50,685 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: -1, nextIndex: 0 01:49:50,685 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:50,685 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:50,780 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1195081643] 01:49:50,780 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : default 01:49:50,780 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-304630517] 01:49:50,780 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@69c6b130: creating a new transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:49:50,782 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} 01:49:50,782 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:49:50,782 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:49:50,782 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1195081643] 01:49:50,782 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} shard 1 01:49:50,782 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:49:50,782 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolving shard 0 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} 01:49:50,782 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 0 to shard default 01:49:50,782 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 0 to shard default 01:49:50,782 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1195081643] 01:49:50,782 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} shard 0 01:49:50,782 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:49:50,783 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:49:50,783 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=OPEN} allocated sequence 0 01:49:50,783 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@69c6b130 readied transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:49:50,783 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=FLUSHED} sending request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1308007628], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@669df06a 01:49:50,783 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} scheduling timeout in 30 seconds 01:49:50,783 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1308007628], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@35fbf0fd 01:49:50,783 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:49:50,783 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:50,783 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1438078722, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6a466917] 01:49:50,784 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:49:50,784 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-304630517, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@68e5c85f] 01:49:50,784 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1438078722, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6a466917] 01:49:50,784 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-304630517, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@68e5c85f] 01:49:50,784 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@d4d02de, attempting to connect 01:49:50,784 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard default resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@25880a69, attempting to connect 01:49:50,784 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} is not yet known 01:49:50,784 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} is not yet known 01:49:50,784 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, nanosAgo=11170, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:49:50,784 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, nanosAgo=11650, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:49:50,784 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:49:50,784 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:49:50,785 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolved shard 0 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-304630517], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present} 01:49:50,785 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-304630517], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 01:49:50,785 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6670a578 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5bcba92c 01:49:50,785 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} previous state FLUSHED 01:49:50,785 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5bcba92c 01:49:50,785 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:49:50,785 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=OPEN} 01:49:50,785 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1308007628], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=OPEN} 01:49:50,785 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1308007628], modifications=1, protocol=THREE_PHASE} 01:49:50,785 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} allocated sequence 0 01:49:50,785 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1308007628], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@669df06a 01:49:50,785 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-304630517], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} scheduling timeout in 30 seconds 01:49:50,785 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2d22e601 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1308007628], coordinated=true}, enqueuedTicks=737118387308} 01:49:50,785 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@69c6b130 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-304630517], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 01:49:50,785 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} 01:49:50,785 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@2b26b840 01:49:50,785 PM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-default-testSnapshotOnRootOverwrite: allocating new ready transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:49:50,785 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5518a3f7 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@7ad95c58, splicing queue 01:49:50,785 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5518a3f7 queue spliced 0 messages 01:49:50,786 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-304630517], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} in 948.8 μs 01:49:50,786 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1438078722], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:49:50,786 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1438078722], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:49:50,786 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 initiating canCommit 01:49:50,786 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:49:50,786 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4b25119a instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4071afec 01:49:50,786 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:49:50,786 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 validated 01:49:50,786 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:50,786 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4071afec 01:49:50,786 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 completed canCommit 01:49:50,786 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:49:50,786 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=OPEN} 01:49:50,786 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1308007628], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=OPEN} 01:49:50,786 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1308007628], modifications=1, protocol=THREE_PHASE} 01:49:50,786 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} allocated sequence 0 01:49:50,786 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1308007628], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@35fbf0fd 01:49:50,786 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1438078722], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:49:50,786 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7e974220 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1308007628], coordinated=true}, enqueuedTicks=737118533280} 01:49:50,786 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@69c6b130 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1438078722], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:49:50,786 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} 01:49:50,786 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@33d49cd1 01:49:50,786 PM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSnapshotOnRootOverwrite: allocating new ready transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:49:50,786 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3286628a superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1b52cd29, splicing queue 01:49:50,786 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3286628a queue spliced 0 messages 01:49:50,786 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 initiating canCommit 01:49:50,786 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:49:50,786 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 validated 01:49:50,786 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:50,787 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1438078722], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 666.7 μs 01:49:50,787 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 completed canCommit 01:49:50,787 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1308007628], coordinated=true} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0}} 01:49:50,787 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1308007628], coordinated=true}, enqueuedTicks=737118387308, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0}} 01:49:50,787 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} canCommit completed 01:49:50,787 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1308007628], coordinated=true} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0}} 01:49:50,787 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1308007628], coordinated=true}, enqueuedTicks=737118533280, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0}} 01:49:50,787 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 01:49:50,787 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} allocated sequence 1 01:49:50,787 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@269a0030 01:49:50,787 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} 01:49:50,787 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} 01:49:50,787 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} allocated sequence 1 01:49:50,787 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} sending request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@269a0030 01:49:50,787 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-304630517], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 01:49:50,787 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2d22e601 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]}, enqueuedTicks=737122305712} 01:49:50,787 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:49:50,787 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@2f8dba29 01:49:50,787 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} 01:49:50,787 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} 01:49:50,787 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} allocated sequence 1 01:49:50,787 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@2f8dba29 01:49:50,787 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1438078722], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:49:50,787 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7e974220 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]}, enqueuedTicks=737122456464} 01:49:50,787 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 initiating preCommit 01:49:50,787 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:49:50,787 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 candidate ready 01:49:50,787 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:49:50,787 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - messages: [] 01:49:50,787 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:49:50,787 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 initiating preCommit 01:49:50,787 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 prepared 01:49:50,787 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:50,787 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:49:50,787 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 completed preCommit 01:49:50,787 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 candidate ready 01:49:50,787 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1: canCommit - candidate rootPath: / 01:49:50,788 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1}} 01:49:50,788 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]}, enqueuedTicks=737122305712, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1}} 01:49:50,788 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1: canCommit - messages: [] 01:49:50,788 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:49:50,788 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 01:49:50,788 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 prepared 01:49:50,788 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:50,788 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 completed preCommit 01:49:50,788 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1}} 01:49:50,788 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]}, enqueuedTicks=737122456464, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1}} 01:49:50,788 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:49:50,788 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} allocated sequence 2 01:49:50,788 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@58c44ddb 01:49:50,788 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} 01:49:50,788 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} 01:49:50,788 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} allocated sequence 2 01:49:50,788 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} sending request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@58c44ddb 01:49:50,788 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-304630517], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 01:49:50,788 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2d22e601 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]}, enqueuedTicks=737123407030} 01:49:50,788 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:49:50,788 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@2ea94d73 01:49:50,788 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} 01:49:50,788 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 initiating commit 01:49:50,788 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} 01:49:50,788 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:49:50,788 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} allocated sequence 2 01:49:50,788 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@2ea94d73 01:49:50,788 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1438078722], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:49:50,788 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7e974220 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]}, enqueuedTicks=737123644764} 01:49:50,788 PM [cluster-test-shard-dispatcher-25] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 113 01:49:50,788 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 01:49:50,788 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 01:49:50,789 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:50,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 initiating commit 01:49:50,789 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:50,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:49:50,789 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 01:49:50,789 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}]]] 01:49:50,789 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 submitted to persistence 01:49:50,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 283 01:49:50,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 01:49:50,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 01:49:50,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:50,789 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 01:49:50,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:50,790 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:49:50,790 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}]]] 01:49:50,790 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 submitted to persistence 01:49:50,790 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}]]] 01:49:50,790 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:49:50,790 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:49:50,790 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=113} 01:49:50,790 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 01:49:50,790 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 1 01:49:50,790 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:50,791 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 01:49:50,791 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-default-testSnapshotOnRootOverwrite status sync done true 01:49:50,791 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}]]] 01:49:50,791 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:49:50,791 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 01:49:50,791 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:49:50,791 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=283} 01:49:50,791 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 01:49:50,791 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 1 01:49:50,791 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:50,791 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 0, nextIndex: 1 01:49:50,791 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 01:49:50,791 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:49:50,791 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 0 01:49:50,791 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 01:49:50,791 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 0 data Simple{size=113} 01:49:50,791 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-cars-testSnapshotOnRootOverwrite status sync done true 01:49:50,791 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:49:50,792 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-1-shard-default-testSnapshotOnRootOverwrite: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@40970d8f}, purged=MutableUnsignedLongSet{size=0}} 01:49:50,792 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-default-testSnapshotOnRootOverwrite: Committed transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:49:50,792 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:49:50,792 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:49:50,792 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2}} 01:49:50,792 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-default-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-575409266] with name member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher 01:49:50,792 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]}, enqueuedTicks=737123407030, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2}} 01:49:50,792 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-575409266] 01:49:50,792 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} doCommit completed 01:49:50,792 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6670a578 completing transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} 01:49:50,792 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR}: initiating purge 01:49:50,792 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} allocated sequence 3 01:49:50,792 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:49:50,792 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@5f901a21 01:49:50,792 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:49:50,792 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} 01:49:50,792 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:50,792 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} 01:49:50,792 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED}: initiating purge 01:49:50,792 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=DONE} allocated sequence 3 01:49:50,792 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-575409266] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:50,792 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 01:49:50,792 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@48b52716 01:49:50,792 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] and replicatedToAllIndex is -1 01:49:50,792 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 69.35 μs 01:49:50,792 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-304630517], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 01:49:50,792 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=4, lastIndex=0, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:49:50,792 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 01:49:50,792 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2d22e601 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]}, enqueuedTicks=737127090211} 01:49:50,793 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[FrontendHistoryMetadata{historyId=0, cookie=0, closed=false, closedTransactions={0=true}, purgedTransactions=ImmutableUnsignedLongSet{size=0}}], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:49:50,793 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: purging transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:49:50,793 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 01:49:50,793 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 01:49:50,793 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:49:50,793 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:49:50,793 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=4, lastAppliedIndex=0, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@28484c84, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:49:50,793 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 0 01:49:50,793 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 0 01:49:50,793 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 0 and term: 4 01:49:50,793 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:49:50,793 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 0 01:49:50,793 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 01:49:50,793 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 01:49:50,793 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 0 data Simple{size=113} 01:49:50,793 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 0, nextIndex: 1 01:49:50,793 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:49:50,793 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:49:50,793 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 0 01:49:50,793 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 01:49:50,793 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 0 data Simple{size=283} 01:49:50,793 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:49:50,794 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 01:49:50,794 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 01:49:50,794 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-1-shard-cars-testSnapshotOnRootOverwrite: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@20e304af}, purged=MutableUnsignedLongSet{size=0}} 01:49:50,794 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-default-testSnapshotOnRootOverwrite: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@34a377db}, purged=MutableUnsignedLongSet{size=0}} 01:49:50,794 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSnapshotOnRootOverwrite: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} 01:49:50,794 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-default-testSnapshotOnRootOverwrite: Committed transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:49:50,794 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSnapshotOnRootOverwrite: Committed transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:49:50,794 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1: commit - successfulFromPrevious: [] 01:49:50,794 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 01:49:50,794 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:49:50,794 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-default-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-1756953592] with name member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher 01:49:50,794 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:49:50,794 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-803338263] with name member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher 01:49:50,794 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 01:49:50,794 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-1756953592] 01:49:50,794 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] and replicatedToAllIndex is -1 01:49:50,794 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=4, lastIndex=0, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:49:50,794 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 3 01:49:50,794 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:49:50,794 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[FrontendHistoryMetadata{historyId=0, cookie=0, closed=false, closedTransactions={0=true}, purgedTransactions=ImmutableUnsignedLongSet{size=0}}], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:49:50,794 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:49:50,794 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 19.21 μs 01:49:50,794 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:50,794 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2}} 01:49:50,794 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]}, enqueuedTicks=737123644764, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2}} 01:49:50,794 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 01:49:50,794 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-1756953592] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:50,794 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 01:49:50,794 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4b25119a completing transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} 01:49:50,794 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] and replicatedToAllIndex is -1 01:49:50,794 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 01:49:50,794 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 01:49:50,794 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@44103930 01:49:50,794 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=4, lastIndex=0, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:49:50,794 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 01:49:50,794 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} 01:49:50,794 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} 01:49:50,794 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED}: initiating purge 01:49:50,794 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=DONE} allocated sequence 3 01:49:50,794 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@1b0e7281 01:49:50,794 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[FrontendHistoryMetadata{historyId=0, cookie=0, closed=false, closedTransactions={0=true}, purgedTransactions=ImmutableUnsignedLongSet{size=0}}], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:49:50,794 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@69c6b130: creating a new snapshot member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 01:49:50,794 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 01:49:50,794 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:49:50,795 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=4, lastAppliedIndex=0, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@55d0718, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:49:50,795 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 0 01:49:50,795 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 0 and term: 4 01:49:50,795 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a journal response 01:49:50,795 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-803338263] 01:49:50,795 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:49:50,795 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-1 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 01:49:50,795 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 60.49 μs 01:49:50,795 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-803338263] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:50,795 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 01:49:50,795 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 01:49:50,795 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 3 01:49:50,795 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:49:50,795 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 01:49:50,795 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=4, lastAppliedIndex=0, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@761b9e8e, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:49:50,795 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 0 01:49:50,795 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 0 and term: 4 01:49:50,795 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1438078722], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:49:50,795 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7e974220 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]}, enqueuedTicks=737129432599} 01:49:50,795 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 01:49:50,795 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 0 01:49:50,795 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:49:50,795 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 0 01:49:50,795 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 01:49:50,795 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 0 data Simple{size=283} 01:49:50,795 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: purging transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:49:50,795 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 01:49:50,795 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 01:49:50,795 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:49:50,795 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:49:50,796 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:49:50,796 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 01:49:50,796 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-cars-testSnapshotOnRootOverwrite: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@7aafcee}, purged=MutableUnsignedLongSet{size=0}} 01:49:50,796 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 01:49:50,796 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-1-shard-cars-testSnapshotOnRootOverwrite 01:49:50,796 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 01:49:50,796 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSnapshotOnRootOverwrite: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} 01:49:50,796 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSnapshotOnRootOverwrite: Committed transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:49:50,796 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-1-shard-default-testSnapshotOnRootOverwrite 01:49:50,796 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#2017042795] with name member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher 01:49:50,796 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onShutDown 01:49:50,796 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating leader transfer 01:49:50,796 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:49:50,796 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2171b47e 01:49:50,796 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 01:49:50,796 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] and replicatedToAllIndex is -1 01:49:50,796 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2171b47e 01:49:50,796 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#2017042795] 01:49:50,796 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=4, lastIndex=0, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:49:50,796 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Attempting to transfer leadership 01:49:50,796 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 3 01:49:50,796 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:50,796 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:49:50,796 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:49:50,796 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[FrontendHistoryMetadata{historyId=0, cookie=0, closed=false, closedTransactions={0=true}, purgedTransactions=ImmutableUnsignedLongSet{size=0}}], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:49:50,796 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}]]] 01:49:50,796 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 48.23 μs 01:49:50,796 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 01:49:50,797 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 01:49:50,797 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSnapshotOnRootOverwrite - waiting on 2 shards 01:49:50,797 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onShutDown 01:49:50,797 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating leader transfer 01:49:50,797 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@638a14dd 01:49:50,797 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@638a14dd 01:49:50,797 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Attempting to transfer leadership 01:49:50,797 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:50,797 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 01:49:50,797 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}]]] 01:49:50,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 01:49:50,797 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 01:49:50,797 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 01:49:50,797 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-default-testSnapshotOnRootOverwrite, matchIndex: 0, lastIndex: 1, isVoting: true 01:49:50,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:50,797 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:49:50,797 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=4, lastAppliedIndex=0, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@14aa7bdf, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:49:50,797 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 0 01:49:50,797 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:49:50,797 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 01:49:50,797 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 0 and term: 4 01:49:50,797 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 01:49:50,797 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a journal response 01:49:50,797 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7e974220 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@4c565b4, splicing queue 01:49:50,796 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#2017042795] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:50,797 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1438078722], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:49:50,797 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7e974220 queue spliced 1 messages 01:49:50,797 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1438078722], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1438078722], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:49:50,797 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 01:49:50,797 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 3 01:49:50,797 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: refreshing backend for shard 1 01:49:50,797 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1438078722], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:49:50,797 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 01:49:50,798 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:49:50,798 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:49:50,798 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1195081643] 01:49:50,798 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 01:49:50,798 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2d22e601 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@2f4c2548, splicing queue 01:49:50,798 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:49:50,798 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-304630517], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} scheduling timeout in 30 seconds 01:49:50,798 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSnapshotOnRootOverwrite, matchIndex: 0, lastIndex: 1, isVoting: true 01:49:50,798 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2d22e601 queue spliced 1 messages 01:49:50,798 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:50,798 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-304630517], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-304630517], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 01:49:50,798 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: refreshing backend for shard 0 01:49:50,798 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 01:49:50,798 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-304630517], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present} 01:49:50,798 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 0 to shard default 01:49:50,798 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 0 to shard default 01:49:50,798 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1195081643] 01:49:50,798 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:49:50,798 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 01:49:50,804 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}]]] 01:49:50,804 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:49:50,804 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:49:50,804 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56} 01:49:50,804 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 01:49:50,804 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 1 01:49:50,804 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:50,804 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}]]] 01:49:50,804 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:49:50,804 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:49:50,804 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57} 01:49:50,804 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 01:49:50,804 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 1 01:49:50,804 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:50,804 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 01:49:50,804 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite] 01:49:50,804 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 01:49:50,804 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 01:49:50,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 01:49:50,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite] 01:49:50,805 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 01:49:50,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 01:49:50,805 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 01:49:50,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:50,805 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 01:49:50,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:50,805 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 1, nextIndex: 2 01:49:50,805 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:49:50,805 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 1 01:49:50,805 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 01:49:50,805 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57} 01:49:50,805 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: replication of member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@50ec3520 01:49:50,805 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSnapshotOnRootOverwrite: finished purging transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:49:50,805 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 1, nextIndex: 2 01:49:50,806 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSnapshotOnRootOverwrite: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} 01:49:50,806 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3}} not found in inflight queue, checking pending queue 01:49:50,806 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:49:50,806 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSnapshotOnRootOverwrite: Purged transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:49:50,806 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3}} 01:49:50,806 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 1 01:49:50,806 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 01:49:50,806 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]}, enqueuedTicks=737129432599, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3}} 01:49:50,806 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:50,806 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56} 01:49:50,806 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=DONE}: purge completed 01:49:50,806 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4071afec purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=DONE} 01:49:50,806 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: replication of member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@5a0fd328 01:49:50,806 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSnapshotOnRootOverwrite, matchIndex: 1, lastIndex: 1, isVoting: true 01:49:50,806 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:49:50,806 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Follower's log matches - sending ElectionTimeout 01:49:50,806 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4b25119a purged transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} 01:49:50,806 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:50,806 PM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-1-shard-default-testSnapshotOnRootOverwrite: finished purging transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:49:50,806 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Leader transfer complete 01:49:50,806 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 01:49:50,806 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3}} not found in inflight queue, checking pending queue 01:49:50,806 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3}} 01:49:50,806 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-default-testSnapshotOnRootOverwrite: Purged transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:49:50,806 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]}, enqueuedTicks=737127090211, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3}} 01:49:50,806 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=DONE}: purge completed 01:49:50,806 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5bcba92c purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=DONE} 01:49:50,806 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR}: purge completed 01:49:50,806 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6670a578 purged transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} 01:49:50,806 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:50,806 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-default-testSnapshotOnRootOverwrite, matchIndex: 1, lastIndex: 1, isVoting: true 01:49:50,806 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Follower's log matches - sending ElectionTimeout 01:49:50,806 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:50,806 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Leader transfer complete 01:49:50,806 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 01:49:50,807 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 1 01:49:50,807 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:49:50,807 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 1 01:49:50,807 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 01:49:50,807 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57} 01:49:50,807 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSnapshotOnRootOverwrite: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} 01:49:50,807 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSnapshotOnRootOverwrite: Purged transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:49:50,807 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 01:49:50,808 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 1 01:49:50,808 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:49:50,808 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 1 01:49:50,808 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 01:49:50,808 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56} 01:49:50,808 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-default-testSnapshotOnRootOverwrite: Purged transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:49:50,808 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 01:49:51,208 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All shards shutdown - sending PoisonPill to self 01:49:51,208 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-testSnapshotOnRootOverwrite: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1438078722]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:49:51,208 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received ShardNotInitializedTimeout message for shard cars 01:49:51,209 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Returning NoShardLeaderException for shard cars 01:49:51,209 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testSnapshotOnRootOverwrite currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:49:51,209 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testSnapshotOnRootOverwrite currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:49:51,209 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testSnapshotOnRootOverwrite currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:49:51,213 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testSnapshotOnRootOverwrite currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:49:51,213 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSnapshotOnRootOverwrite 01:49:51,213 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageSlicer - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader)_slicer-id-603: Closing 01:49:51,213 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-default-testSnapshotOnRootOverwrite 01:49:51,213 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-default-testSnapshotOnRootOverwrite (Leader)_slicer-id-602: Closing 01:49:51,213 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 01:49:51,213 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 01:49:51,214 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-testSnapshotOnRootOverwrite 01:49:51,214 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 01:49:51,214 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:49:51,214 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager$ShardNotInitializedTimeout] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1195081643] to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1195081643] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1195081643] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:51,214 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: shutting down 01:49:51,214 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 01:49:51,214 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 01:49:51,215 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:51,215 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:51,220 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:51,220 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:51,220 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:51,220 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:51,230 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSnapshotOnRootOverwrite: Closing 01:49:51,230 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSnapshotOnRootOverwrite 01:49:51,230 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Closing 01:49:51,230 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 01:49:51,231 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-default-testSnapshotOnRootOverwrite 01:49:51,231 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Closing 01:49:51,231 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 01:49:51,231 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testSnapshotOnRootOverwrite 01:49:51,231 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:51,231 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:52,232 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-2421982606466426472] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:52,246 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:52,246 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:52,247 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:52.248] [cluster-test-akka.actor.internal-dispatcher-6] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:52,253 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [5990704085422017429] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:52,256 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:52,256 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:52,259 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:52.259] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:52,269 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:52,275 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [1413441365391789251] 01:49:52,276 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:52,277 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:52,277 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:52,279 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:52,279 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:52,287 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:52,287 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:52,287 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:52,289 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:52,294 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-967215392993027657] 01:49:52,294 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:52,303 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:52,303 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:52,303 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:52,303 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:52,310 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:52,321 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-6062700459403098888] 01:49:52,321 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:52,322 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:52,322 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:52,325 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:52,325 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:52,333 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:52,335 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:52,336 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 01:49:52,345 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 01:49:52,346 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#9239850] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:52,350 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 01:49:52,350 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-1-shard-cars-testWriteTransactionWithSingleShard 01:49:52,350 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:52,350 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:49:52,350 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: no initial generation override, starting from 0 01:49:52,350 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:49:52,350 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@22868f30 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7f34c614 01:49:52,352 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:52,360 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:49:52,362 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3a365796 01:49:52,362 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1710164992350, None)) 01:49:52,362 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164992349,0,0)) 01:49:52,362 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:52,362 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7f34c614 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@529bcffb 01:49:52,362 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4ff59cbf 01:49:52,362 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} started 01:49:52,363 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:52,364 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:52,364 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:52,366 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 01:49:52,367 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:52,367 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testWriteTransactionWithSingleShard 01:49:52,367 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:49:52,367 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:49:52,367 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard on shard member-1-shard-cars-testWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800] 01:49:52,367 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-cars-testWriteTransactionWithSingleShard, persistent : true 01:49:52,368 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 01:49:52,368 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:52,368 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-notifier#-2128773403 created and ready for shard:member-1-shard-cars-testWriteTransactionWithSingleShard 01:49:52,368 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-notifier#-2128773403] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:52,368 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-member-1:snapshot-read#578929953] 01:49:52,368 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-member-1:snapshot-read#578929953] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:52,369 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:52,369 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:52,369 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:49:52,369 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 01:49:52,369 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 01:49:52,369 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-1-shard-cars-testWriteTransactionWithSingleShard is initialized 01:49:52,369 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 01:49:52,369 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard 01:49:52,370 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-1607601032] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:52,371 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 01:49:52,371 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from null to Follower 01:49:52,372 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: no initial generation override, starting from 0 01:49:52,372 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:49:52,372 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1bcb2918 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@476bd708 01:49:52,372 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@43545b3d 01:49:52,372 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1710164992372, None)) 01:49:52,372 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164992371,0,0)) 01:49:52,372 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@476bd708 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@371afaa9 01:49:52,373 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} started 01:49:52,379 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 01:49:52,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testWriteTransactionWithSingleShard 01:49:52,379 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:52,379 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:49:52,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@c878514 01:49:52,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:52,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithSingleShard 01:49:52,379 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:49:52,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:49:52,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard on shard member-2-shard-cars-testWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#1908353862] 01:49:52,380 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithSingleShard, persistent : true 01:49:52,380 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#1908353862] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:52,380 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-1721981497 created and ready for shard:member-2-shard-cars-testWriteTransactionWithSingleShard 01:49:52,380 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-1721981497] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:52,380 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-1961438520] 01:49:52,380 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 01:49:52,380 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-1961438520] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:52,381 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:52,381 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:52,381 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 01:49:52,381 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithSingleShard is initialized 01:49:52,381 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:49:52,381 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 01:49:52,381 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 01:49:52,381 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:49:52,381 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard 01:49:52,381 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testWriteTransactionWithSingleShard from null to Follower 01:49:52,423 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#9239850] 01:49:52,423 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 01:49:52,423 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800] 01:49:52,678 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:52,678 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 01:49:52,678 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testWriteTransactionWithSingleShard] 01:49:52,678 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:49:52,678 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 1 01:49:52,678 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:52,678 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 01:49:52,678 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:49:52,678 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 01:49:52,678 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 01:49:52,678 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from Follower to Candidate 01:49:52,987 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:52,997 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:49:52,997 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:49:52,997 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 2 01:49:53,268 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:49:53,268 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:49:53,268 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 3 01:49:53,289 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:49:53,289 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:53,289 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:53,289 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard 01:49:53,289 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:49:53,289 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:53,289 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:49:53,289 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:49:53,289 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-2-shard-cars-testWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:49:53,294 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:53,294 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard 01:49:53,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:49:53,294 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:53,294 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:49:53,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:49:53,336 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:49:53,517 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:49:53,517 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:49:53,517 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 4 01:49:53,517 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testWriteTransactionWithSingleShard 01:49:53,518 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:53,518 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Cluster state: [] 01:49:53,518 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:53,519 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:53,519 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 01:49:53,519 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:53,519 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:49:53,519 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:49:53,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:49:53,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:49:53,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testWriteTransactionWithSingleShard] 01:49:53,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:53,520 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:49:53,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:49:53,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: became leader with frontend state for [] 01:49:53,520 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 01:49:53,520 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:49:53,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:53,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:53,520 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from Candidate to Leader 01:49:53,520 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 01:49:53,521 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard: Closing 01:49:53,521 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testWriteTransactionWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:49:53,521 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:49:53,521 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:53,521 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:53,521 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 01:49:53,521 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithSingleShard status sync done false 01:49:53,522 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: -1, nextIndex: 0 01:49:53,522 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:53,530 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3b501a09: creating a new transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:49:53,530 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} 01:49:53,530 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:49:53,530 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:49:53,530 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-1607601032] 01:49:53,530 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} shard 1 01:49:53,530 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:49:53,530 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:49:53,531 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3b501a09 readied transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:49:53,531 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=4, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@599fbe24 01:49:53,531 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:49:53,531 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:53,531 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard, primaryVersion=13] 01:49:53,531 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard, primaryVersion=13] 01:49:53,531 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@3825c043, attempting to connect 01:49:53,532 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} is not yet known 01:49:53,532 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, nanosAgo=11210, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:49:53,533 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:49:53,533 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:49:53,533 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:53,533 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@51164f69 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@13d3170b 01:49:53,533 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:49:53,533 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@13d3170b 01:49:53,533 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:53,533 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 01:49:53,533 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=4, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 01:49:53,533 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:49:53,534 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=4, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2258/0x00000008016bac68@63be85ac 01:49:53,534 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:49:53,534 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@68debd9f transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=4, protocol=SIMPLE}, enqueuedTicks=739865951652} 01:49:53,534 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-606: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=4, protocol=SIMPLE}} 01:49:53,534 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-606: Message size: 508 bytes, total slices to send: 1 01:49:53,534 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-606: Message does not need to be sliced - sending original message 01:49:53,534 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3b501a09 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:53,534 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 01:49:53,534 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@475c36b7 01:49:53,534 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4f88e137 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@50db9fdb, splicing queue 01:49:53,534 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4f88e137 queue spliced 0 messages 01:49:53,534 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.327 ms 01:49:53,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:53,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: readying transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:53,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 ready 01:49:53,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: transitioned member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 to ready 01:49:53,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct canCommit 01:49:53,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:53,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 validated 01:49:53,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:53,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct preCommit 01:49:53,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:53,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 candidate ready 01:49:53,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1: canCommit - candidate rootPath: / 01:49:53,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1: canCommit - messages: [] 01:49:53,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 01:49:53,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 prepared 01:49:53,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:53,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct commit 01:49:53,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:53,538 PM [cluster-test-shard-dispatcher-30] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 246 01:49:53,538 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 01:49:53,538 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 01:49:53,538 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:53,538 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:53,538 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 01:49:53,538 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}]]] 01:49:53,538 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 submitted to persistence 01:49:53,538 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 01:49:53,539 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}]]] 01:49:53,539 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 01:49:53,539 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:49:53,539 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=246} 01:49:53,539 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 01:49:53,539 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 1 01:49:53,539 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:53,539 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 01:49:53,539 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithSingleShard status sync done true 01:49:53,539 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 0, nextIndex: 1 01:49:53,539 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:49:53,539 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 0 01:49:53,539 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 01:49:53,539 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 0 data Simple{size=246} 01:49:53,539 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:53,540 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-1-shard-cars-testWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@6a823369}, purged=MutableUnsignedLongSet{size=0}} 01:49:53,540 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:49:53,540 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:53,540 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1: commit - successfulFromPrevious: [] 01:49:53,540 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:49:53,540 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-1759450621] with name member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher 01:49:53,540 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-1759450621] 01:49:53,540 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:49:53,540 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:53,540 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 01:49:53,540 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 80.53 μs 01:49:53,540 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-1759450621] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:53,541 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 0 01:49:53,541 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=4, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0}} 01:49:53,541 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:49:53,541 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 0 01:49:53,541 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=4, protocol=SIMPLE}, enqueuedTicks=739865951652, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0}} 01:49:53,541 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 01:49:53,541 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=4, protocol=SIMPLE} completed with TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0} 01:49:53,541 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 0 data Simple{size=246} 01:49:53,541 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} directCommit completed 01:49:53,541 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 01:49:53,541 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:49:53,541 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3b501a09: creating a new snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 01:49:53,541 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#266057884]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@6d886809 01:49:53,541 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} 01:49:53,541 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:53,541 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SEALED}: initiating purge 01:49:53,541 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 01:49:53,541 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} allocated sequence 1 01:49:53,541 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:49:53,541 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#266057884]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@4bc71281 01:49:53,541 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@5630cf9c 01:49:53,541 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:53,541 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@68debd9f transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#266057884]}, enqueuedTicks=739876289901} 01:49:53,541 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@68debd9f transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=739876492434} 01:49:53,542 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@2d957060}, purged=MutableUnsignedLongSet{size=0}} 01:49:53,542 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:49:53,542 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:53,542 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#1447645058] with name member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher 01:49:53,542 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#1447645058] 01:49:53,542 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:49:53,542 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 01:49:53,542 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 46.02 μs 01:49:53,542 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#1447645058] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:53,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: purging transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:53,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 01:49:53,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 01:49:53,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:49:53,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1 01:49:53,543 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-605: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 01:49:53,543 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-605: Message size: 391 bytes, total slices to send: 1 01:49:53,543 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-605: Message does not need to be sliced - sending original message 01:49:53,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 01:49:53,544 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 01:49:53,544 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=739876492434, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 01:49:53,545 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0} 01:49:53,545 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@606bdb9e: creating a new snapshot member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:49:53,545 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} 01:49:53,545 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:49:53,545 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:49:53,545 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#9239850] 01:49:53,545 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} shard 1 01:49:53,545 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:49:53,545 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:49:53,545 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1873533611], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@685a5652 01:49:53,545 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:49:53,546 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:53,546 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@52e32fda] 01:49:53,546 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@52e32fda] 01:49:53,546 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6d32c14b, attempting to connect 01:49:53,546 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} is not yet known 01:49:53,546 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, nanosAgo=5650, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:49:53,546 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:49:53,546 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:49:53,546 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:49:53,546 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@46eb963f instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@419b24f3 01:49:53,546 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:49:53,546 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@419b24f3 01:49:53,546 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:53,546 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 01:49:53,546 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1873533611], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 01:49:53,546 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@606bdb9e finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:49:53,546 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 01:49:53,546 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@18f37d1d 01:49:53,546 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7caf4595 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@640b4a4f, splicing queue 01:49:53,546 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7caf4595 queue spliced 0 messages 01:49:53,546 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 338.2 μs 01:49:53,547 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0} 01:49:53,547 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3b501a09: creating a new transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:49:53,547 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:49:53,547 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=OPEN} allocated sequence 0 01:49:53,547 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3b501a09 readied transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:49:53,547 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@7fb0c85d 01:49:53,547 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@68debd9f transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=1, protocol=SIMPLE}, enqueuedTicks=739882102697} 01:49:53,547 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-606: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, message: Envelope{sessionId=0, txSequence=3, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=1, protocol=SIMPLE}} 01:49:53,547 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-606: Message size: 416 bytes, total slices to send: 1 01:49:53,547 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-606: Message does not need to be sliced - sending original message 01:49:53,548 PM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:49:53,548 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: readying transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:49:53,548 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 ready 01:49:53,548 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: transitioned member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 to ready 01:49:53,548 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 initiating direct canCommit 01:49:53,548 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:49:53,548 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 validated 01:49:53,548 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:53,548 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 initiating direct preCommit 01:49:53,548 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:49:53,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 candidate ready 01:49:53,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1: canCommit - candidate rootPath: / 01:49:53,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1: canCommit - messages: [] 01:49:53,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1: preCommit - successfulFromPrevious: [] 01:49:53,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 prepared 01:49:53,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:53,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 initiating direct commit 01:49:53,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:49:53,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 213 01:49:53,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 01:49:53,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 01:49:53,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:53,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:49:53,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 01:49:53,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}]]] 01:49:53,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 submitted to persistence 01:49:53,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 01:49:53,550 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}]]] 01:49:53,550 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 2 01:49:53,550 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:49:53,550 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65} 01:49:53,550 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 01:49:53,550 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=213} 01:49:53,550 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 01:49:53,550 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 3 01:49:53,550 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:53,550 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 01:49:53,550 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 01:49:53,550 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 2, nextIndex: 3 01:49:53,550 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 01:49:53,550 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 1 01:49:53,550 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 01:49:53,550 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65} 01:49:53,550 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: replication of member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@1a9382d4 01:49:53,550 PM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: finished purging transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:53,551 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:49:53,551 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:53,551 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 2 01:49:53,551 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 01:49:53,551 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 2 data Simple{size=213} 01:49:53,551 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:49:53,551 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:49:53,551 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:49:53,551 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1: commit - successfulFromPrevious: [] 01:49:53,551 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 changing state from COMMIT_PENDING to COMMITTED 01:49:53,551 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:49:53,551 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 1 for term 4 01:49:53,551 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:49:53,551 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=2 01:49:53,551 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 107.8 μs 01:49:53,551 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#266057884]} with Envelope{sessionId=0, txSequence=1, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 01:49:53,551 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#266057884]}, enqueuedTicks=739876289901, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 01:49:53,551 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE}: purge completed 01:49:53,551 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@13d3170b purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} 01:49:53,551 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:49:53,551 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@51164f69 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 01:49:53,551 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=1, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0}} 01:49:53,551 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=1, protocol=SIMPLE}, enqueuedTicks=739882102697, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0}} 01:49:53,551 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=FLUSHED} directCommit completed 01:49:53,551 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=FLUSHED}: initiating purge 01:49:53,551 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=DONE} allocated sequence 1 01:49:53,552 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#266057884]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@1e7b68d2 01:49:53,552 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3b501a09: creating a new snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 01:49:53,552 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:53,552 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 01:49:53,552 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@68debd9f transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#266057884]}, enqueuedTicks=739886765124} 01:49:53,552 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} allocated sequence 0 01:49:53,552 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 2 01:49:53,552 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} sending request ExistsTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1240/0x00000008013035b8@74334316 01:49:53,552 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 01:49:53,552 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 1 01:49:53,552 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 01:49:53,552 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@68debd9f transmitting entry ConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]}, enqueuedTicks=739886982567} 01:49:53,552 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65} 01:49:53,552 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:49:53,552 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:53,552 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 2 01:49:53,552 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 01:49:53,552 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 2 data Simple{size=213} 01:49:53,552 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:49:53,552 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:49:53,552 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:49:53,552 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 1 for term 4 01:49:53,552 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=2 01:49:53,552 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:49:53,553 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 104.5 μs 01:49:53,553 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: purging transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:49:53,553 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}] 01:49:53,553 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}] 01:49:53,553 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:49:53,553 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 01:49:53,553 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}]]] 01:49:53,553 PM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1 01:49:53,554 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}] 01:49:53,554 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}]]] 01:49:53,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 01:49:53,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:49:53,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66} 01:49:53,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}] 01:49:53,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 2 01:49:53,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:53,555 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request ExistsTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} with Envelope{sessionId=0, txSequence=5, message=ExistsTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, exists=true}} 01:49:53,555 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]}, enqueuedTicks=739886982567, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=ExistsTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, exists=true}} 01:49:53,555 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteProxyTransaction - Exists request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] completed with ExistsTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, exists=true} 01:49:53,555 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3b501a09: creating a new snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 01:49:53,555 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 01:49:53,555 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} allocated sequence 0 01:49:53,555 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@16466215 01:49:53,555 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@68debd9f transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=739890314942} 01:49:53,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}] 01:49:53,555 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 3, nextIndex: 4 01:49:53,555 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 01:49:53,555 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 3 01:49:53,555 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 2 to 3 01:49:53,555 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66} 01:49:53,555 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: replication of member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@5baa50d4 01:49:53,555 PM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: finished purging transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:49:53,555 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:49:53,555 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:49:53,555 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 2 for term 4 01:49:53,555 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:49:53,555 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=3 01:49:53,556 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#266057884]} with Envelope{sessionId=0, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=1}} 01:49:53,556 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#266057884]}, enqueuedTicks=739886765124, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=1}} 01:49:53,556 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=DONE}: purge completed 01:49:53,556 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@13d3170b purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=DONE} 01:49:53,556 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 3 01:49:53,556 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 01:49:53,556 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 3 01:49:53,556 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 2 to 3 01:49:53,556 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66} 01:49:53,556 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:49:53,556 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:49:53,556 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 2 for term 4 01:49:53,556 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=3 01:49:53,557 PM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1 01:49:53,557 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-605: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, message: Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0}} 01:49:53,557 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-605: Message size: 369 bytes, total slices to send: 1 01:49:53,557 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-605: Message does not need to be sliced - sending original message 01:49:53,558 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0}} 01:49:53,558 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#266057884], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=739890314942, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0}} 01:49:53,558 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0} 01:49:53,558 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@606bdb9e: creating a new snapshot member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 01:49:53,558 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1 for transaction member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 01:49:53,558 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#9239850] 01:49:53,558 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 01:49:53,558 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800] 01:49:53,558 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800] 01:49:53,609 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:49:53,610 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-cars-testWriteTransactionWithSingleShard 01:49:53,610 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader)_slicer-id-608: Closing 01:49:53,610 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:49:53,610 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 01:49:53,612 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:53,612 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:53,620 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:53,620 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:53,620 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:53,620 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:53,621 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:53,630 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:49:53,630 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithSingleShard 01:49:53,630 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 01:49:53,630 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:49:53,630 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 01:49:53,631 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:53,631 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:54,626 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [1413441365391789251] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:54,646 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-967215392993027657] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:54,649 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:54,649 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:54,651 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:54.652] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:54,664 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:49:54,670 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://reinstated-member2@127.0.0.1:2559] with UID [2426740520656091441] 01:49:54,670 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:54,671 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:54,671 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Started up successfully 01:49:54,671 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:54,671 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:54,680 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:54,681 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:54,683 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 01:49:54,685 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#781134794] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,685 PM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 01:49:54,686 PM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: recovered identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:49:54,686 PM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 01:49:54,686 PM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 01:49:54,687 PM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5bbfd0e0 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7e92babc 01:49:54,687 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testWriteTransactionWithSingleShard 01:49:54,687 PM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5bc9c56f 01:49:54,687 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-people-testWriteTransactionWithSingleShard 01:49:54,687 PM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1710164994687, None)) 01:49:54,687 PM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164994686,0,0)) 01:49:54,687 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@15847778 01:49:54,687 PM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7e92babc to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7adca087 01:49:54,687 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} started 01:49:54,687 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:54,687 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithSingleShard 01:49:54,688 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testWriteTransactionWithSingleShard 01:49:54,688 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithSingleShard, persistent : true 01:49:54,688 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#944889225] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,688 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Shard created : member-2-shard-people-testWriteTransactionWithSingleShard, persistent : true 01:49:54,688 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 01:49:54,688 PM [reinstated-member2-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-1086141719 created and ready for shard:member-2-shard-cars-testWriteTransactionWithSingleShard 01:49:54,688 PM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#491145014] 01:49:54,688 PM [reinstated-member2-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-632546633] 01:49:54,689 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard#-262902257] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,689 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-1086141719] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,689 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#491145014] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,689 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-632546633] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,690 PM [reinstated-member2-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard-notifier#-1710765015 created and ready for shard:member-2-shard-people-testWriteTransactionWithSingleShard 01:49:54,690 PM [reinstated-member2-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard-notifier#-1710765015] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,690 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://reinstated-member2/deadLetters] 01:49:54,690 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 01:49:54,690 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://reinstated-member2/deadLetters] 01:49:54,690 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:49:54,690 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 01:49:54,690 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 0, size: 246 01:49:54,690 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 01:49:54,690 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-2-shard-people-testWriteTransactionWithSingleShard with journal batch size 1 01:49:54,690 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 0 to 0 01:49:54,690 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@21d156a1}, purged=MutableUnsignedLongSet{size=0}} 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 1, size: 65 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 2, size: 213 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 1 to 2 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 2 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 3, size: 66 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 3 to 3 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 2 to 3 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://reinstated-member2/deadLetters] 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Recovery completed in 1.002 ms - Switching actor to Follower - last log index = 3, last log term = 4, snapshot index = -1, snapshot term = -1, journal size = 4 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithSingleShard is initialized 01:49:54,691 PM [reinstated-member2-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Received TimeoutNow - switching to Candidate 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [] 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=5, votedFor=member-2-shard-cars-testWriteTransactionWithSingleShard 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 5 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:49:54,691 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 01:49:54,691 PM [reinstated-member2-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 01:49:54,695 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://reinstated-member2/deadLetters] 01:49:54,695 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:54,695 PM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:49:54,695 PM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [] 01:49:54,695 PM [reinstated-member2-shard-dispatcher-18] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:49:54,695 PM [reinstated-member2-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:49:54,695 PM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: became leader with frontend state for [member-2-frontend-datastore-testWriteTransactionWithSingleShard] 01:49:54,695 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:49:54,695 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithSingleShard is no longer the leader 01:49:54,696 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Follower): Received TimeoutNow - switching to Candidate 01:49:54,696 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [] 01:49:54,696 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-2-shard-people-testWriteTransactionWithSingleShard 01:49:54,696 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [people] 01:49:54,696 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Starting new election term 1 01:49:54,696 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-testWriteTransactionWithSingleShard is initialized 01:49:54,696 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:54,696 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithSingleShard (Follower): Closing 01:49:54,696 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:49:54,696 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithSingleShard is no longer the leader 01:49:54,696 PM [reinstated-member2-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 01:49:54,696 PM [reinstated-member2-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from null to Follower 01:49:54,696 PM [reinstated-member2-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard 01:49:54,696 PM [reinstated-member2-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 01:49:54,696 PM [reinstated-member2-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:49:54,696 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:54,696 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:54,696 PM [reinstated-member2-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testWriteTransactionWithSingleShard from Candidate to Leader 01:49:54,699 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:49:54,699 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [] 01:49:54,699 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 01:49:54,699 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:49:54,699 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: became leader with frontend state for [] 01:49:54,699 PM [reinstated-member2-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 01:49:54,699 PM [reinstated-member2-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , registered listener akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard 01:49:54,699 PM [reinstated-member2-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithSingleShard, leaderId=member-2-shard-people-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:49:54,699 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:49:54,699 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:49:54,699 PM [reinstated-member2-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-people-testWriteTransactionWithSingleShard from Candidate to Leader 01:49:54,699 PM [reinstated-member2-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 01:49:54,737 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#781134794] 01:49:54,738 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 01:49:54,738 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#944889225] 01:49:54,739 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@448b7517: creating a new snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 01:49:54,739 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} 01:49:54,739 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:49:54,739 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:49:54,739 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#781134794] 01:49:54,739 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, history=0, cookie=0} shard 1 01:49:54,739 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 01:49:54,740 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} allocated sequence 0 01:49:54,740 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=0, replyTo=Actor[akka://reinstated-member2/user/$a#-406643174], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@6460b4b3 01:49:54,740 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} scheduling timeout in 30 seconds 01:49:54,740 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:54,740 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#944889225, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4afe9ce] 01:49:54,740 PM [reinstated-member2-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#944889225, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4afe9ce] 01:49:54,741 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6d010168, attempting to connect 01:49:54,741 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: retiring state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, nanosAgo=45530820, purgedHistories=MutableUnsignedLongSet{size=0}}, outdated by request from client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 01:49:54,741 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, nanosAgo=6710, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 01:49:54,741 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:49:54,741 PM [reinstated-member2-akka.actor.default-dispatcher-19] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#944889225], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:49:54,742 PM [reinstated-member2-akka.actor.default-dispatcher-19] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#944889225], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:49:54,742 PM [reinstated-member2-akka.actor.default-dispatcher-19] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3f79b987 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@538c7c27 01:49:54,742 PM [reinstated-member2-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} previous state OPEN 01:49:54,742 PM [reinstated-member2-akka.actor.default-dispatcher-19] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@538c7c27 01:49:54,742 PM [reinstated-member2-akka.actor.default-dispatcher-19] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1 01:49:54,742 PM [reinstated-member2-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} 01:49:54,742 PM [reinstated-member2-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=0, replyTo=Actor[akka://reinstated-member2/user/$a#-406643174], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} 01:49:54,742 PM [reinstated-member2-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@448b7517 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#944889225], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:49:54,742 PM [reinstated-member2-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} 01:49:54,742 PM [reinstated-member2-akka.actor.default-dispatcher-19] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@6fb99b1b 01:49:54,742 PM [reinstated-member2-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@27529f4f superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@178fb2c6, splicing queue 01:49:54,742 PM [reinstated-member2-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@27529f4f queue spliced 0 messages 01:49:54,742 PM [reinstated-member2-akka.actor.default-dispatcher-19] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#944889225], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 649.2 μs 01:49:54,742 PM [reinstated-member2-akka.actor.default-dispatcher-19] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=0} 01:49:54,743 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 01:49:54,743 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-cars-testWriteTransactionWithSingleShard 01:49:54,743 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-people-testWriteTransactionWithSingleShard 01:49:54,743 PM [reinstated-member2-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithSingleShard - waiting on 2 shards 01:49:54,743 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onShutDown 01:49:54,743 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@13306bbb 01:49:54,743 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onShutDown 01:49:54,743 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@13306bbb 01:49:54,743 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@370cc2c9 01:49:54,743 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardDataTree - member-2-shard-people-testWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@370cc2c9 01:49:54,743 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithSingleShard 01:49:54,743 PM [reinstated-member2-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-people-testWriteTransactionWithSingleShard 01:49:54,743 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] MessageSlicer - member-2-shard-cars-testWriteTransactionWithSingleShard (Leader)_slicer-id-612: Closing 01:49:54,743 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] MessageSlicer - member-2-shard-people-testWriteTransactionWithSingleShard (Leader)_slicer-id-613: Closing 01:49:54,743 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:49:54,744 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:49:54,744 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 01:49:54,744 PM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 01:49:54,744 PM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:49:54,744 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:49:54,744 PM [main] [WARN] ActorUtils - ShardManager for testWriteTransactionWithSingleShard data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:49:54,745 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:49:54,744 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#781134794] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#781134794] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,745 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:49:54,745 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: shutting down 01:49:54,745 PM [main] [WARN] ActorUtils - ShardManager for testWriteTransactionWithSingleShard data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:49:54,745 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:49:54,745 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:49:54,746 PM [reinstated-member2-akka.actor.default-dispatcher-19] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#781134794] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#781134794] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,746 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://reinstated-member2/user/$a#-406643174] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://reinstated-member2/user/$a#-406643174] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,746 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:54,746 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:54,749 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:54,757 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:54,757 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:54,759 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:54.760] [reinstated-member2-akka.actor.internal-dispatcher-2] [Cluster(akka://reinstated-member2)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:54,769 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:54,775 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-5568969669461460559] 01:49:54,775 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:54,777 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:54,777 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:54,782 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:54,782 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:54,786 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:54,786 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:54,786 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:54,791 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:54,800 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-4115841535779974268] 01:49:54,800 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:54,803 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:54,803 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:54,803 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:54,803 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:54,816 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:54,823 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-5188570527851199053] 01:49:54,823 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:54,824 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:54,824 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:54,832 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:54,833 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:54,833 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:54,833 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:54,835 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,841 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-134637299] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,841 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,844 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: no initial generation override, starting from 0 01:49:54,844 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:49:54,844 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6f5aca18 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@233dd431 01:49:54,844 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2702448e 01:49:54,844 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, 0, 1710164994844, None)) 01:49:54,844 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,845 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Creating local shard: member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,845 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164994843,0,0)) 01:49:54,845 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:54,845 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@233dd431 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4a93def6 01:49:54,845 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,845 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@23720c51 01:49:54,845 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} started 01:49:54,845 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:54,845 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,845 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:54,847 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:54,848 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,850 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,850 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,850 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: updated peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040] 01:49:54,850 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, persistent : true 01:49:54,850 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,851 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-1:snapshot-read#1101340777] 01:49:54,851 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#-537898714 created and ready for shard:member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,851 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#-537898714] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,851 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-1:snapshot-read#1101340777] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,851 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with journal batch size 1 01:49:54,855 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:54,855 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:54,855 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:49:54,855 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 01:49:54,855 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from null to Follower 01:49:54,855 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:54,856 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Initializing shard [cars] 01:49:54,856 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is initialized 01:49:54,856 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,856 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from null to Follower 01:49:54,858 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,858 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#592768030] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,859 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:54,863 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: no initial generation override, starting from 0 01:49:54,863 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,863 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:49:54,863 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Creating local shard: member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,864 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2f0d625e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4b885b6a 01:49:54,864 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@cda9348 01:49:54,864 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, 0, 1710164994864, None)) 01:49:54,864 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164994863,0,0)) 01:49:54,864 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:54,864 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,864 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4b885b6a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@508cf2a9 01:49:54,865 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} started 01:49:54,865 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6cd42fc 01:49:54,865 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:54,865 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,865 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,865 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,865 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: updated peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-2136210198] 01:49:54,865 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, persistent : true 01:49:54,866 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with journal batch size 1 01:49:54,866 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-2136210198] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,866 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-2:snapshot-read#1053527302] 01:49:54,866 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-2:snapshot-read#1053527302] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,866 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#739585057 created and ready for shard:member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,866 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#739585057] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,867 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:54,867 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:54,867 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Initializing shard [cars] 01:49:54,867 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is initialized 01:49:54,867 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:49:54,867 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 01:49:54,867 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,867 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from null to Follower 01:49:54,867 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 01:49:54,867 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from null to Follower 01:49:54,872 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:54,873 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:49:54,915 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-134637299] 01:49:54,915 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: findLocalShard : cars 01:49:54,915 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040] 01:49:55,137 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:55,137 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Received ElectionTimeout - switching to Candidate 01:49:55,137 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 01:49:55,137 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:55,137 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 1 01:49:55,137 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:55,137 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 01:49:55,137 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:49:55,137 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 01:49:55,137 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Follower to Candidate 01:49:55,138 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Follower to Candidate 01:49:55,416 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:55,448 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:49:55,448 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:55,448 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 2 01:49:55,767 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:49:55,767 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:55,768 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 3 01:49:55,798 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:49:55,798 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:55,799 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:55,799 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:55,799 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:55,799 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:55,799 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:55,799 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:55,799 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:55,804 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:55,804 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:55,804 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:55,804 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:55,804 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:55,804 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:55,843 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:49:55,997 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:49:55,997 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:55,997 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 4 01:49:55,997 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:55,999 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:55,999 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Cluster state: [] 01:49:55,999 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:55,999 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:55,999 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=null 01:49:55,999 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:55,999 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:55,999 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:49:56,0 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:49:56,0 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:49:56,1 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 01:49:56,1 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:56,1 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:49:56,1 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:49:56,1 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: became leader with frontend state for [] 01:49:56,1 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Candidate to Leader 01:49:56,1 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderPayloadVersion=13] 01:49:56,1 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:56,1 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:56,1 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Candidate to Leader 01:49:56,1 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All Shards are ready - data store testTransactionWithCreateTxFailureDueToNoLeader is ready 01:49:56,2 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:49:56,2 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:56,2 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader status sync done false 01:49:56,2 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderPayloadVersion=13] 01:49:56,2 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:56,2 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:56,2 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All Shards are ready - data store testTransactionWithCreateTxFailureDueToNoLeader is ready 01:49:56,3 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader updated: matchIndex: -1, nextIndex: 0 01:49:56,3 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:56,22 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4b4e780e: creating a new snapshot member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 01:49:56,22 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} 01:49:56,22 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:49:56,22 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:49:56,22 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#592768030] 01:49:56,22 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:56,22 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, history=0, cookie=0} shard 1 01:49:56,22 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, primaryVersion=13] 01:49:56,22 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 01:49:56,22 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, primaryVersion=13] 01:49:56,22 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:49:56,22 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-730618116], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@1bd09559 01:49:56,22 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} scheduling timeout in 2 seconds 01:49:56,22 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@3741d5ab, attempting to connect 01:49:56,24 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} is not yet known 01:49:56,24 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, nanosAgo=10930, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:49:56,25 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:49:56,25 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:49:56,25 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:56,25 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4a22123c instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2c671622 01:49:56,25 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:49:56,25 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2c671622 01:49:56,25 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1 01:49:56,25 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} 01:49:56,26 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-730618116], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} 01:49:56,26 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:49:56,26 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-730618116], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1248/0x0000000801301d10@5082b321 01:49:56,26 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1996670856 nanoseconds 01:49:56,26 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2b160c14 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-730618116], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=742357558073} 01:49:56,26 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4b4e780e finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:56,26 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} 01:49:56,26 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@3f54c952 01:49:56,26 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@62ad8459 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@27f32896, splicing queue 01:49:56,26 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@62ad8459 queue spliced 0 messages 01:49:56,26 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 782.4 μs 01:49:56,27 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: allocating new open snapshot member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1 01:49:56,28 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-615: slice: identifier: member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0}} 01:49:56,28 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-615: Message size: 239 bytes, total slices to send: 1 01:49:56,28 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-615: Message does not need to be sliced - sending original message 01:49:56,29 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-730618116], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0}} 01:49:56,29 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-730618116], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=742357558073, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0}} 01:49:56,29 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0} 01:49:56,29 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:49:56,29 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:56,29 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader)_slicer-id-618: Closing 01:49:56,29 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Aborting 0 pending queued transactions 01:49:56,30 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:49:56,30 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:56,30 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:56,35 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:56,35 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:56,36 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:56,36 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:56,37 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:56,51 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 01:49:56,152 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithCreateTxFailureDueToNoLeader 01:49:56,152 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4b4e780e: creating a new transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 01:49:56,152 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 01:49:56,152 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:56,152 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:49:56,152 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4b4e780e readied transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 01:49:56,152 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 01:49:56,152 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:56,152 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-730618116], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@5e12e885 01:49:56,152 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 01:49:56,152 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : true 01:49:56,152 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:56,152 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 01:49:56,152 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2b160c14 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-730618116], modifications=1, protocol=SIMPLE}, enqueuedTicks=742487585442} 01:49:56,152 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-616: slice: identifier: member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-730618116], modifications=1, protocol=SIMPLE}} 01:49:56,153 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-616: Message size: 402 bytes, total slices to send: 1 01:49:56,153 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-616: Message does not need to be sliced - sending original message 01:49:56,356 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 01:49:56,357 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 01:49:56,357 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 01:49:56,357 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Received ElectionTimeout - switching to Candidate 01:49:56,357 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 01:49:56,357 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:56,357 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 5 01:49:56,357 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:56,357 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:49:56,357 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 01:49:56,357 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:49:56,357 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 01:49:56,357 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:49:56,357 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Follower to Candidate 01:49:56,357 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=null, leaderPayloadVersion=-1] 01:49:56,358 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:56,358 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Follower to Candidate 01:49:56,358 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:49:56,358 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2b160c14 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@3c9ef6d0, splicing queue 01:49:56,358 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1794460547 nanoseconds 01:49:56,358 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2b160c14 queue spliced 1 messages 01:49:56,358 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:56,358 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: refreshing backend for shard 1 01:49:56,358 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:49:56,358 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:49:56,358 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:49:56,358 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#592768030] 01:49:56,358 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:56,358 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Candidate, leaderId=null, activeMember=true] 01:49:56,547 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:49:56,547 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=6, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:56,547 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 6 01:49:56,547 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:56,577 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received ShardNotInitializedTimeout message for shard cars 01:49:56,577 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Returning NoShardLeaderException for shard cars 01:49:56,577 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:49:56,578 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:49:56,578 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:49:56,578 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:49:56,726 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:49:56,726 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=7, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:56,726 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 7 01:49:56,726 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=7, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:56,936 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:49:56,937 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=8, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:56,937 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 8 01:49:56,937 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=8, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:57,43 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-5568969669461460559] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:57,47 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-5568969669461460559] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:57,147 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:49:57,147 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=9, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:57,147 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 9 01:49:57,147 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=9, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:57,296 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:49:57,297 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=10, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:57,297 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 10 01:49:57,297 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=10, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:57,476 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:49:57,476 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=11, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:57,476 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 11 01:49:57,477 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=11, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:57,597 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:57,597 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:49:57,597 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:49:57,597 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#592768030] 01:49:57,597 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:57,597 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Candidate, leaderId=null, activeMember=true] 01:49:57,646 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:49:57,646 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=12, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:57,646 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 12 01:49:57,646 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=12, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:57,817 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received ShardNotInitializedTimeout message for shard cars 01:49:57,817 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Returning NoShardLeaderException for shard cars 01:49:57,817 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:49:57,817 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=13, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:57,817 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 13 01:49:57,817 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=13, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:57,817 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:49:57,818 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:49:57,818 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:49:57,818 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:49:58,7 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:49:58,7 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=14, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:58,7 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 14 01:49:58,7 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=14, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:58,36 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: running timer on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} 01:49:58,36 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} is empty 01:49:58,36 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: not scheduling timeout on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} 01:49:58,36 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: running timer on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:58,36 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} is empty 01:49:58,36 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: not scheduling timeout on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:58,177 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: running timer on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:58,177 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} timed out entry TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-730618116], modifications=1, protocol=SIMPLE}, enqueuedTicks=742487585442, sessionId=0, txSequence=1} 01:49:58,179 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} timed out 1 tasks 01:49:58,179 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: not scheduling timeout on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:58,182 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=FLUSHED} directCommit completed 01:49:58,182 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=FLUSHED}: initiating purge 01:49:58,182 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=DONE} allocated sequence 1 01:49:58,182 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-730618116]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@3bd408f2 01:49:58,182 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} has a successor, not scheduling timer 01:49:58,182 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1999886759 nanoseconds 01:49:58,183 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:49:58,183 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithCreateTxFailureDueToNoLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:49:58,183 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:49:58,183 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:49:58,184 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithCreateTxFailureDueToNoLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:49:58,184 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:49:58,184 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:49:58,184 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:49:58,185 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Aborting 0 pending queued transactions 01:49:58,185 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:49:58,185 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:58,185 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:59,199 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:59,199 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:59,200 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:59.202] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:59,203 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-4115841535779974268] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:59,209 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:59,209 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:59,211 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:59.212] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:59,219 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:59,223 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-776027980858096425] 01:49:59,224 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:59,225 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:59,225 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:59,225 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:59,225 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:59,235 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:59,235 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:59,238 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:59,243 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1786592304418957598] 01:49:59,243 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:59,245 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:59,245 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:59,246 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:59,257 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:59,257 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:59,259 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5bcba92c aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0, state=OPEN} 01:49:59,259 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@65524165 01:49:59,259 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-304630517], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 01:49:59,259 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-304630517], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 01:49:59,259 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4071afec aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-1, state=OPEN} 01:49:59,259 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1308007628]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@520bd78e 01:49:59,259 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1438078722], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:49:59,259 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1438078722], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:49:59,259 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0} 01:49:59,259 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@13d3170b aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} 01:49:59,259 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} allocated sequence 1 01:49:59,259 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@32e49eb0 01:49:59,259 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:49:59,259 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@68debd9f transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=0, protocol=ABORT}, enqueuedTicks=745594614083} 01:49:59,259 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-606: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=0, protocol=ABORT}} 01:49:59,259 PM [Cleaner-1] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-606: Message size: 285 bytes, total slices to send: 1 01:49:59,260 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-606: Message does not need to be sliced - sending original message 01:49:59,260 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0} 01:49:59,260 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@46eb963f aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1873533611], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@609333a5 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} 01:49:59,260 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@419b24f3 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1873533611]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@2b4200ba 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:49:59,260 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@13d3170b aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} allocated sequence 1 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@4e376126 01:49:59,260 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@68debd9f transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=0, protocol=ABORT}, enqueuedTicks=745595043610} 01:49:59,260 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-606: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, message: Envelope{sessionId=0, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=0, protocol=ABORT}} 01:49:59,260 PM [Cleaner-1] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-606: Message size: 285 bytes, total slices to send: 1 01:49:59,260 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-606: Message does not need to be sliced - sending original message 01:49:59,260 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0} 01:49:59,260 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@13d3170b aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} allocated sequence 1 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@43276847 01:49:59,260 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@68debd9f transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=0, protocol=ABORT}, enqueuedTicks=745595162202} 01:49:59,260 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-606: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, message: Envelope{sessionId=0, txSequence=9, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#266057884], modifications=0, protocol=ABORT}} 01:49:59,260 PM [Cleaner-1] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-606: Message size: 285 bytes, total slices to send: 1 01:49:59,260 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-606: Message does not need to be sliced - sending original message 01:49:59,260 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0} 01:49:59,260 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@419b24f3 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1873533611]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@71ac5b1f 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#354635800], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:49:59,260 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3f79b987 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=1, replyTo=Actor[akka://reinstated-member2/user/$a#-406643174], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@72367f6e 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} 01:49:59,260 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@538c7c27 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=0, replyTo=Actor[akka://reinstated-member2/user/$a#-406643174]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@60bb1648 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#944889225], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:49:59,260 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4a22123c aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-730618116], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@635fcc31 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} has a successor, not scheduling timer 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SEALED} allocated sequence 1 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-730618116], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x00000008013037f0@703817e6 01:49:59,260 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-81982040], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} has a successor, not scheduling timer 01:49:59,260 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0} 01:49:59,268 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:59,273 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [944818923670065813] 01:49:59,273 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:59,274 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:59,274 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:59,282 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:59,282 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:59,285 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:59,286 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:59,287 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithMultipleShards 01:49:59,294 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:59,296 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#1215993542] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:59,296 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 01:49:59,298 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithMultipleShards 01:49:59,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:49:59,298 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: no initial generation override, starting from 0 01:49:59,298 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:49:59,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:49:59,298 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1644365f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7f3b3b34 01:49:59,298 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:59,298 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7b04a00b 01:49:59,298 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:49:59,298 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:49:59,299 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, 0, 1710164999299, None)) 01:49:59,299 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164999298,0,0)) 01:49:59,299 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7f3b3b34 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@61329dcd 01:49:59,299 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@22d89938 01:49:59,299 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} started 01:49:59,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:59,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:49:59,300 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:59,301 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:59,302 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:49:59,302 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:49:59,302 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#635682364] 01:49:59,302 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:49:59,302 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:49:59,302 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:49:59,302 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards on shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-1888082314] 01:49:59,303 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, persistent : true 01:49:59,303 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithMultipleShards 01:49:59,303 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-people-testChainedTransactionFailureWithMultipleShards, persistent : true 01:49:59,304 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-people-testChainedTransactionFailureWithMultipleShards with journal batch size 1 01:49:59,304 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#635682364] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:59,304 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-1888082314] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:59,304 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-478456638] 01:49:59,305 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#637123489] 01:49:59,305 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#1387779020 created and ready for shard:member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:49:59,305 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-754115966] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:59,305 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:59,305 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-938031354] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:59,307 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with journal batch size 1 01:49:59,307 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 01:49:59,307 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-754115966 created and ready for shard:member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:49:59,307 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:59,307 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:59,308 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-478456638] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:59,308 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#1387779020] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:59,308 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#637123489] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:59,309 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [people] 01:49:59,309 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is initialized 01:49:59,310 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:59,310 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:49:59,310 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 01:49:59,310 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:59,310 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:59,310 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 01:49:59,310 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [cars] 01:49:59,310 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is initialized 01:49:59,310 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from null to Follower 01:49:59,312 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:49:59,314 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:59,314 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:49:59,314 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithMultipleShards 01:49:59,314 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:49:59,314 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: no initial generation override, starting from 0 01:49:59,314 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:49:59,314 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:49:59,314 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1a3e0e88 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@49bfb6e 01:49:59,314 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@478b76ea 01:49:59,314 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:59,314 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:49:59,314 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:49:59,315 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 01:49:59,315 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 01:49:59,315 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, 0, 1710164999315, None)) 01:49:59,315 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164999314,0,0)) 01:49:59,315 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from null to Follower 01:49:59,315 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@49bfb6e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@53af2c70 01:49:59,315 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6aa1273b 01:49:59,315 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} started 01:49:59,316 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:59,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:59,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:49:59,316 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:49:59,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:49:59,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#-258542002] 01:49:59,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:49:59,316 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:49:59,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:49:59,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards on shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards#2066135817] 01:49:59,316 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, persistent : true 01:49:59,317 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with journal batch size 1 01:49:59,317 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-people-testChainedTransactionFailureWithMultipleShards, persistent : true 01:49:59,317 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-people-testChainedTransactionFailureWithMultipleShards with journal batch size 1 01:49:59,317 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#-258542002] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:59,317 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards#2066135817] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:59,317 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#-1563964161] 01:49:59,317 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-2016083045 created and ready for shard:member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:49:59,317 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#1543682868] 01:49:59,318 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-659007610] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:59,318 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-2016083045] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:59,318 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#-1563964161] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:59,318 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#1543682868] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:59,318 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-659007610 created and ready for shard:member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:49:59,319 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:59,319 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:59,319 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:59,319 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:59,319 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:59,319 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:49:59,319 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [people] 01:49:59,319 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:49:59,319 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards is initialized 01:49:59,319 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:59,319 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:49:59,319 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [cars] 01:49:59,319 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards is initialized 01:49:59,319 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 01:49:59,319 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 01:49:59,319 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 01:49:59,319 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:49:59,319 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 01:49:59,319 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-2-shard-people-testChainedTransactionFailureWithMultipleShards from null to Follower 01:49:59,320 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards from null to Follower 01:49:59,366 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#1215993542] 01:49:59,366 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: findLocalShard : cars 01:49:59,366 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#635682364] 01:49:59,567 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:59,567 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:49:59,567 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testChainedTransactionFailureWithMultipleShards] 01:49:59,567 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:49:59,567 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 1 01:49:59,567 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:59,567 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Closing 01:49:59,567 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:59,567 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:49:59,567 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from Follower to Candidate 01:49:59,567 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from Follower to Candidate 01:49:59,607 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:59,607 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:49:59,607 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testChainedTransactionFailureWithMultipleShards] 01:49:59,608 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:49:59,608 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 1 01:49:59,608 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:59,608 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Closing 01:49:59,608 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:59,608 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:49:59,608 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from Follower to Candidate 01:49:59,608 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from Follower to Candidate 01:49:59,817 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:49:59,817 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:49:59,817 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 2 01:49:59,877 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:49:59,877 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:49:59,877 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 2 01:49:59,886 PM [cluster-test-shard-dispatcher-30] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:59,896 PM [cluster-test-shard-dispatcher-30] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:00,107 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:50:00,107 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:00,108 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 3 01:50:00,187 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:50:00,187 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:00,187 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 3 01:50:00,238 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:00,238 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:00,238 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:00,238 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:00,238 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:00,238 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:00,238 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:00,238 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:00,238 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:00,238 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:00,238 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:00,238 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:00,238 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:00,238 PM [cluster-test-shard-dispatcher-34] [INFO] TestShard - Peer address for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:00,246 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:00,246 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:00,246 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:00,246 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:00,246 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:00,246 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:00,246 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:00,246 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:00,246 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:00,246 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:00,303 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:00,417 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:50:00,417 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:00,417 PM [cluster-test-shard-dispatcher-35] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 4 01:50:00,417 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:00,419 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:00,419 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Cluster state: [] 01:50:00,420 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:00,420 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:00,420 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=null 01:50:00,420 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:00,420 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:00,420 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:00,421 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:00,421 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:00,421 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testChainedTransactionFailureWithMultipleShards] 01:50:00,421 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:00,421 PM [cluster-test-shard-dispatcher-35] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:00,421 PM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:50:00,421 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: became leader with frontend state for [] 01:50:00,422 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from Candidate to Leader 01:50:00,422 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 01:50:00,422 PM [cluster-test-shard-dispatcher-35] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:00,422 PM [cluster-test-shard-dispatcher-35] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:00,422 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from Candidate to Leader 01:50:00,422 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Closing 01:50:00,422 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testChainedTransactionFailureWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:00,422 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 01:50:00,422 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:00,422 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:00,422 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards status sync done false 01:50:00,423 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:50:00,423 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:00,427 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:50:00,427 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:00,427 PM [cluster-test-shard-dispatcher-35] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 4 01:50:00,427 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:00,432 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:00,432 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Cluster state: [] 01:50:00,432 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:00,432 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:00,432 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=null 01:50:00,432 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:00,432 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:00,432 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:00,433 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:00,433 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:00,433 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testChainedTransactionFailureWithMultipleShards] 01:50:00,433 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:00,433 PM [cluster-test-shard-dispatcher-35] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:00,433 PM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:50:00,433 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: became leader with frontend state for [] 01:50:00,433 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from Candidate to Leader 01:50:00,433 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 01:50:00,433 PM [cluster-test-shard-dispatcher-35] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:00,433 PM [cluster-test-shard-dispatcher-35] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:00,433 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from Candidate to Leader 01:50:00,433 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All Shards are ready - data store testChainedTransactionFailureWithMultipleShards is ready 01:50:00,434 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Closing 01:50:00,434 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testChainedTransactionFailureWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:00,434 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 01:50:00,434 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:00,434 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:00,434 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All Shards are ready - data store testChainedTransactionFailureWithMultipleShards is ready 01:50:00,434 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-people-testChainedTransactionFailureWithMultipleShards status sync done false 01:50:00,434 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:50:00,435 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:00,472 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#1215993542] 01:50:00,472 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: findLocalShard : people 01:50:00,472 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-1888082314] 01:50:00,473 PM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3b39c3ca 01:50:00,473 PM [main] [DEBUG] AbstractDOMBroker - Transaction chain 0 created with listener Mock for DOMTransactionChainListener, hashCode: 612371751, backing store chains {CONFIGURATION=org.opendaylight.controller.cluster.databroker.ClientBackedTransactionChain@3da28b70} 01:50:00,473 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3b39c3ca changed state from IDLE to TX_OPEN 01:50:00,473 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} 01:50:00,473 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard people 01:50:00,473 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard people 01:50:00,473 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-938031354] 01:50:00,473 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:50:00,473 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} shard 1 01:50:00,474 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 01:50:00,474 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:50:00,474 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 01:50:00,474 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 01:50:00,474 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 01:50:00,474 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} 01:50:00,474 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard cars 01:50:00,474 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard cars 01:50:00,474 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-938031354] 01:50:00,474 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} shard 2 01:50:00,474 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:00,474 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} scheduling timeout in 30 seconds 01:50:00,474 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 01:50:00,474 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 for transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 01:50:00,474 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} allocated sequence 0 01:50:00,474 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 01:50:00,474 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@644a4a11 01:50:00,474 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@2bb21e61 01:50:00,475 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3b39c3ca changed state from TX_OPEN to IDLE 01:50:00,475 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3b39c3ca readied transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 01:50:00,475 PM [main] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 is submitted for execution. 01:50:00,475 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 01:50:00,475 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@2cdeb199 01:50:00,475 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=FLUSHED} allocated sequence 1 01:50:00,475 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@48a7a241 01:50:00,475 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@10cb0b85, attempting to connect 01:50:00,475 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6a19751e, attempting to connect 01:50:00,476 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} is not yet known 01:50:00,476 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, nanosAgo=10471, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:50:00,476 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} is not yet known 01:50:00,477 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, nanosAgo=6050, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:50:00,478 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:50:00,478 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:50:00,482 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-1888082314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent} 01:50:00,482 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-1888082314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} 01:50:00,482 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@461f39c1 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@7654b4f0 01:50:00,482 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:50:00,482 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-1888082314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:50:00,482 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2aae1423 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961]}, enqueuedTicks=746808764653} 01:50:00,482 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@7654b4f0 01:50:00,482 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 01:50:00,482 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} 01:50:00,482 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=1, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} 01:50:00,483 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 01:50:00,483 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2218/0x00000008016a3c10@248b61bb 01:50:00,483 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2aae1423 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=1, protocol=READY}, enqueuedTicks=746809685548} 01:50:00,483 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-622: slice: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=1, protocol=READY}} 01:50:00,483 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-622: Message size: 406 bytes, total slices to send: 1 01:50:00,483 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-622: Message does not need to be sliced - sending original message 01:50:00,483 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=0, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} 01:50:00,483 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 01:50:00,483 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 1 01:50:00,483 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 01:50:00,483 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2219/0x00000008016a6000@463ec7d6 01:50:00,483 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:50:00,483 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2aae1423 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=746809885961} 01:50:00,483 PM [cluster-test-shard-dispatcher-34] [DEBUG] LeaderFrontendState - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 01:50:00,483 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-622: slice: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=0, protocol=THREE_PHASE}} 01:50:00,483 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-622: Message size: 298 bytes, total slices to send: 1 01:50:00,483 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-622: Message does not need to be sliced - sending original message 01:50:00,483 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3b39c3ca finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-1888082314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} 01:50:00,483 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 01:50:00,483 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@42e0a523 01:50:00,484 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@480f0ebf superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@71fb37ce, splicing queue 01:50:00,484 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@480f0ebf queue spliced 0 messages 01:50:00,484 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-1888082314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} in 1.663 ms 01:50:00,484 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#635682364], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent} 01:50:00,484 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#635682364], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} 01:50:00,484 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@aa1b94f instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6519e84d 01:50:00,484 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} previous state FLUSHED 01:50:00,484 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#635682364], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:50:00,484 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55d368a7 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961]}, enqueuedTicks=746809475395} 01:50:00,484 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6519e84d 01:50:00,484 PM [cluster-test-shard-dispatcher-34] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: allocating new open transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 01:50:00,484 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 for transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 01:50:00,484 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 01:50:00,484 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} 01:50:00,484 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@156307a3 01:50:00,484 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=1, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} 01:50:00,484 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} allocated sequence 0 01:50:00,484 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2218/0x00000008016a3c10@63f3bc22 01:50:00,484 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55d368a7 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=1, protocol=READY}, enqueuedTicks=746809737379} 01:50:00,484 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: readying transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 01:50:00,484 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 ready 01:50:00,484 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-622: slice: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, message: Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=1, protocol=READY}} 01:50:00,484 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE}}}]} 01:50:00,484 PM [cluster-test-shard-dispatcher-34] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: transitioned member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 to ready 01:50:00,484 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-622: Message size: 491 bytes, total slices to send: 1 01:50:00,485 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-622: Message does not need to be sliced - sending original message 01:50:00,485 PM [cluster-test-shard-dispatcher-34] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 is already in state READY (READY) 01:50:00,485 PM [cluster-test-shard-dispatcher-34] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 initiating canCommit 01:50:00,485 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Validating transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 01:50:00,485 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=0, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} 01:50:00,485 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 validated 01:50:00,485 PM [cluster-test-shard-dispatcher-34] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:00,485 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 1 01:50:00,485 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2219/0x00000008016a6000@4aaba0 01:50:00,485 PM [cluster-test-shard-dispatcher-34] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 completed canCommit 01:50:00,485 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55d368a7 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=746809938652} 01:50:00,485 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-622: slice: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, message: Envelope{sessionId=1, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=0, protocol=THREE_PHASE}} 01:50:00,485 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-622: Message size: 299 bytes, total slices to send: 1 01:50:00,485 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-622: Message does not need to be sliced - sending original message 01:50:00,485 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3b39c3ca finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#635682364], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} 01:50:00,485 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 01:50:00,485 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@579aa4b7 01:50:00,485 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@22606835 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@17994103, splicing queue 01:50:00,485 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@22606835 queue spliced 0 messages 01:50:00,485 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#635682364], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} in 1.181 ms 01:50:00,485 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 01:50:00,486 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0}} 01:50:00,486 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=1, protocol=READY}, enqueuedTicks=746809685548, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0}} 01:50:00,486 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 01:50:00,486 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0} 01:50:00,486 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 01:50:00,486 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:50:00,486 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1}} 01:50:00,486 PM [cluster-test-shard-dispatcher-34] [DEBUG] LeaderFrontendState - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 01:50:00,486 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=746809885961, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1}} 01:50:00,486 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} canCommit completed 01:50:00,486 PM [cluster-test-shard-dispatcher-34] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: allocating new open transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 01:50:00,486 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 01:50:00,486 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@76b2c282 01:50:00,486 PM [cluster-test-shard-dispatcher-34] [DEBUG] FrontendTransaction - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1216440961], modifications=1, protocol=READY} failed to process org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:00,487 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: request Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1216440961], modifications=1, protocol=READY}} failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1216440961], modifications=1, protocol=READY} failed to process at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:147) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ... 33 more 01:50:00,487 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: request Envelope{sessionId=1, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1216440961], modifications=0, protocol=THREE_PHASE}} failed org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:00,488 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 01:50:00,496 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=1, protocol=READY} with Envelope{sessionId=1, txSequence=1, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1216440961], modifications=1, protocol=READY} failed to process}} 01:50:00,496 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=1, protocol=READY}, enqueuedTicks=746809737379, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1216440961], modifications=1, protocol=READY} failed to process}} 01:50:00,496 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=1, protocol=READY} completed with TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1216440961], modifications=1, protocol=READY} failed to process} 01:50:00,496 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1216440961], modifications=1, protocol=READY} failed to process at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:147) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ... 33 more 01:50:00,496 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=2, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 01:50:00,496 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1216440961], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=746809938652, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 01:50:00,496 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during phase CAN_COMMIT, starting Abort org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:00,496 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DOMBrokerTransactionChain - Transaction chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@196f40d6 failed. TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0]]} at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1127) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) at org.opendaylight.controller.cluster.databroker.actors.dds.VotingFuture.resolveResult(VotingFuture.java:91) at org.opendaylight.controller.cluster.databroker.actors.dds.VotingFuture.voteNo(VotingFuture.java:67) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction.lambda$canCommit$5(AbstractProxyTransaction.java:549) at org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction.lambda$handleReplayedModifyTransactionRequest$17(RemoteProxyTransaction.java:516) at org.opendaylight.controller.cluster.access.client.ConnectionEntry.complete(ConnectionEntry.java:47) at org.opendaylight.controller.cluster.access.client.AbstractClientConnection.receiveResponse(AbstractClientConnection.java:494) at org.opendaylight.controller.cluster.access.client.AbstractReceivingClientConnection.receiveResponse(AbstractReceivingClientConnection.java:68) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onResponse(ClientActorBehavior.java:184) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onRequestFailure(ClientActorBehavior.java:196) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.internalOnRequestFailure(ClientActorBehavior.java:248) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:155) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:48) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.onReceiveCommand(AbstractClientActor.java:74) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) ... 17 more 01:50:00,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:50:00,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} sending request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1216440961]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1246/0x00000008013016a0@663a538b 01:50:00,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} 01:50:00,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 2 01:50:00,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} sending request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1216440961]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1246/0x00000008013016a0@663a538b 01:50:00,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2aae1423 transmitting entry ConnectionEntry{request=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1216440961]}, enqueuedTicks=746832385732} 01:50:00,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 2 01:50:00,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} sending request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1216440961]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1246/0x00000008013016a0@40a5005d 01:50:00,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} 01:50:00,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 2 01:50:00,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} sending request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1216440961]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1246/0x00000008013016a0@40a5005d 01:50:00,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55d368a7 transmitting entry ConnectionEntry{request=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1216440961]}, enqueuedTicks=746832514815} 01:50:00,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} canCommit completed 01:50:00,500 PM [cluster-test-shard-dispatcher-34] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 aborting 01:50:00,501 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: aborting head of queue member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 in state member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 01:50:00,501 PM [cluster-test-shard-dispatcher-34] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1: abort - successfulFromPrevious: [] 01:50:00,502 PM [cluster-test-shard-dispatcher-34] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 aborted 01:50:00,502 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: request Envelope{sessionId=1, txSequence=3, message=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1216440961]}} failed org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:00,506 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1216440961]} with Envelope{sessionId=0, txSequence=3, message=TransactionAbortSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2}} 01:50:00,506 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1216440961]}, enqueuedTicks=746832385732, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionAbortSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2}} 01:50:00,506 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} abort completed 01:50:00,506 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: initiating purge 01:50:00,506 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 3 01:50:00,506 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1216440961]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@7027d8b9 01:50:00,506 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} 01:50:00,506 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED}: initiating purge 01:50:00,506 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 3 01:50:00,506 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1216440961]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@44357b3d 01:50:00,506 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2aae1423 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1216440961]}, enqueuedTicks=746841530293} 01:50:00,507 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1216440961]} with Envelope{sessionId=1, txSequence=3, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 01:50:00,507 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1216440961]}, enqueuedTicks=746832514815, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 01:50:00,507 PM [cluster-test-akka.actor.default-dispatcher-10] [ERROR] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during Abort. org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:00,507 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} abort completed 01:50:00,507 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR}: initiating purge 01:50:00,507 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 3 01:50:00,507 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1216440961]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@65346445 01:50:00,507 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} 01:50:00,507 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED}: initiating purge 01:50:00,507 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} allocated sequence 3 01:50:00,507 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1216440961]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@ab7074d 01:50:00,507 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55d368a7 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1216440961]}, enqueuedTicks=746842365286} 01:50:00,507 PM [main] [DEBUG] DOMBrokerTransactionChain - Chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@196f40d6 is no longer running 01:50:00,507 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:50:00,507 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:00,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: purging transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 01:50:00,508 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:00,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}] 01:50:00,508 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards - waiting on 2 shards 01:50:00,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}] 01:50:00,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:00,508 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onShutDown 01:50:00,508 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Initiating leader transfer 01:50:00,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onShutDown 01:50:00,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Initiating leader transfer 01:50:00,508 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@70baa45a 01:50:00,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@c37287 01:50:00,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 01:50:00,508 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 01:50:00,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@c37287 01:50:00,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Attempting to transfer leadership 01:50:00,508 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@70baa45a 01:50:00,508 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Attempting to transfer leadership 01:50:00,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:00,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:00,508 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:00,508 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:00,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}]]] 01:50:00,508 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:50:00,508 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}]]] 01:50:00,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}] 01:50:00,508 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:00,508 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:00,508 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:50:00,508 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: not currently active leader, rejecting request Envelope{sessionId=1, txSequence=4, message=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1216440961]}}. isLeader: true, isLeaderActive: false,isLeadershipTransferInProgress: true, paused: true 01:50:00,508 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:00,508 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:00,509 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: request Envelope{sessionId=1, txSequence=4, message=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1216440961]}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#635682364] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:00,510 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#635682364], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} indicated no leadership, reconnecting it org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#635682364] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:00,511 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55d368a7 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@18bb3147, splicing queue 01:50:00,511 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#635682364], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:50:00,511 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55d368a7 queue spliced 2 messages 01:50:00,511 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#635682364], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#635682364], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} 01:50:00,511 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards] 01:50:00,511 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: refreshing backend for shard 2 01:50:00,511 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#635682364], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent} 01:50:00,511 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards] 01:50:00,511 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard cars 01:50:00,511 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard cars 01:50:00,511 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-938031354] 01:50:00,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:00,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 01:50:00,511 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:50:00,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:00,511 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:50:00,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:00,511 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 01:50:00,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}]]] 01:50:00,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 2 01:50:00,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:00,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77} 01:50:00,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 01:50:00,511 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 01:50:00,511 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:00,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78} 01:50:00,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}] 01:50:00,511 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2aae1423 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@1844c558, splicing queue 01:50:00,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 2 01:50:00,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:00,511 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-1888082314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:50:00,511 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2aae1423 queue spliced 2 messages 01:50:00,512 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-1888082314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-1888082314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} 01:50:00,512 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: refreshing backend for shard 1 01:50:00,512 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}]]] 01:50:00,512 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-1888082314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent} 01:50:00,512 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 01:50:00,512 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:00,512 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard people 01:50:00,512 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77} 01:50:00,512 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard people 01:50:00,512 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 01:50:00,512 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-938031354] 01:50:00,512 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 1 01:50:00,512 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:00,512 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:50:00,512 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@188102fa, attempting to connect 01:50:00,512 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 01:50:00,512 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 0 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77} 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2220/0x00000008016a65d8@12431bf5 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] LeaderFrontendState - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5b894ce0}, purged=MutableUnsignedLongSet{size=0}} 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 1 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78} 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: replication of member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@53f1706b 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: finished purging transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Purged transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testChainedTransactionFailureWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Leader transfer complete 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 0 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77} 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2220/0x00000008016a65d8@646c83ec 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] LeaderFrontendState - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@10c48779}, purged=MutableUnsignedLongSet{size=0}} 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, matchIndex: 0, lastIndex: 0, isVoting: true 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Leader transfer complete 01:50:00,513 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:50:00,514 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} is not yet known 01:50:00,514 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: not currently leader, rejecting request ConnectClientRequest{target=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testChainedTransactionFailureWithMultipleShards_member-1-shard-cars-testChainedTransactionFailureWithMultipleShards$c], minVersion=POTASSIUM, maxVersion=POTASSIUM}. isLeader: true, isLeaderActive: false,isLeadershipTransferInProgress: true. 01:50:00,514 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} not found in inflight queue, checking pending queue 01:50:00,514 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} 01:50:00,514 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961]}, enqueuedTicks=746808764653, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} 01:50:00,514 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0} 01:50:00,514 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3}} not found in inflight queue, checking pending queue 01:50:00,514 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1216440961]} with Envelope{sessionId=0, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3}} 01:50:00,514 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1216440961]}, enqueuedTicks=746841530293, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3}} 01:50:00,514 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 01:50:00,514 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@7654b4f0 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} 01:50:00,514 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: purge completed 01:50:00,514 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@461f39c1 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 01:50:00,514 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} not found in inflight queue, checking pending queue 01:50:00,514 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961]} with Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} 01:50:00,514 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1216440961]}, enqueuedTicks=746809475395, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} 01:50:00,514 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0} 01:50:00,515 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Connect attempt to cars failed, will retry org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#635682364] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:00,515 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 2 shard cars failed, removing state java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#635682364] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:00,515 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#635682364] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:00,515 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timed out resolving shard 2, scheduling retry in 1 second java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#635682364] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:00,516 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 01:50:00,516 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 01:50:00,516 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}] 01:50:00,516 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 0 01:50:00,516 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:00,516 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 0 01:50:00,516 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 01:50:00,516 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 1 01:50:00,516 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77} 01:50:00,516 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 01:50:00,516 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 0 01:50:00,516 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 01:50:00,516 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@139b1e74}, purged=MutableUnsignedLongSet{size=0}} 01:50:00,516 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 01:50:00,516 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards status sync done true 01:50:00,516 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-people-testChainedTransactionFailureWithMultipleShards status sync done true 01:50:00,516 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:50:00,516 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77} 01:50:00,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5b754e56}, purged=MutableUnsignedLongSet{size=0}} 01:50:00,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 01:50:00,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 1 01:50:00,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 01:50:00,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78} 01:50:00,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Purged transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 01:50:00,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:50:00,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:50:00,927 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All shards shutdown - sending PoisonPill to self 01:50:00,927 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#635682364]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:00,927 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:00,927 PM [cluster-test-shard-dispatcher-35] [INFO] TestShard - Stopping Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:00,927 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader)_slicer-id-626: Closing 01:50:00,927 PM [cluster-test-shard-dispatcher-35] [DEBUG] MessageSlicer - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader)_slicer-id-625: Closing 01:50:00,927 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 01:50:00,927 PM [cluster-test-shard-dispatcher-35] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 01:50:00,927 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 01:50:00,928 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 01:50:00,928 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:50:00,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: shutting down 01:50:00,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 01:50:00,928 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 01:50:00,929 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#1215993542] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#1215993542] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:00,929 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#1215993542] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#1215993542] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:00,929 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#1215993542] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#1215993542] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:00,929 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:00,929 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:00,934 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:00,934 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:00,934 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:00,934 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:00,936 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:00,949 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 01:50:00,949 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:00,949 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:00,949 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Closing 01:50:00,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Closing 01:50:00,949 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 01:50:00,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 01:50:00,949 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 01:50:00,950 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:00,950 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:01,942 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-776027980858096425] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:01,962 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [1786592304418957598] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:01,968 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:01,968 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:01,970 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:01.971] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:01,982 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:01,982 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:01,984 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:01.985] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:01,999 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:02,12 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-8777707846164332390] 01:50:02,12 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:02,14 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:02,14 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:02,21 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:02,21 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:02,26 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:02,26 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:02,26 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:02,30 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:02,36 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [735413509306248000] 01:50:02,36 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:02,37 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:02,37 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:02,48 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:02,48 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:02,51 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:02,65 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-4792705041987168572] 01:50:02,65 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:02,71 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:02,71 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:02,72 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:02,72 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:02,79 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:02,80 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:02,82 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 01:50:02,83 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:02,86 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:02,89 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 01:50:02,90 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1662680583] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:02,92 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 01:50:02,92 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:02,93 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3861513f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4b7e14aa 01:50:02,93 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4f41991d 01:50:02,93 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1710165002093, None)) 01:50:02,93 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165002092,0,0)) 01:50:02,93 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4b7e14aa to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3b0330cc 01:50:02,93 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 01:50:02,94 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:02,95 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:02,97 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 01:50:02,98 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#458819312] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:02,99 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 01:50:02,100 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 01:50:02,100 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:02,100 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-1-shard-people-testLeadershipTransferOnShutdown 01:50:02,100 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 01:50:02,100 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:02,100 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5f3f7b79 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@341c087 01:50:02,100 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:02,100 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:02,100 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:02,100 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4cdf2db2 01:50:02,100 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:02,100 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:02,101 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 01:50:02,101 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:02,101 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-2-shard-people-testLeadershipTransferOnShutdown 01:50:02,101 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:02,101 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:02,101 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:02,101 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@46434e8b 01:50:02,101 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1710165002100, None)) 01:50:02,101 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165002099,0,0)) 01:50:02,102 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7f8c219b 01:50:02,102 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@341c087 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@44a48414 01:50:02,102 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 01:50:02,102 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:02,102 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:02,102 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:02,102 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:02,102 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-2-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#655979141] 01:50:02,102 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testLeadershipTransferOnShutdown 01:50:02,102 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:02,102 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:50:02,102 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-2-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#2108128209] 01:50:02,103 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-people-testLeadershipTransferOnShutdown, persistent : true 01:50:02,103 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-2-shard-cars-testLeadershipTransferOnShutdown, persistent : true 01:50:02,103 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 01:50:02,103 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-2-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 01:50:02,103 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#2108128209] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:02,103 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#655979141] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:02,103 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#1861598234] 01:50:02,104 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:02,104 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:02,104 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-1-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-2018160395] 01:50:02,104 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testLeadershipTransferOnShutdown 01:50:02,104 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:02,104 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:02,104 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-1-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1353936402] 01:50:02,104 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-notifier#1164419373 created and ready for shard:member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:02,104 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-notifier#-1833564683] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:02,104 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-cars-testLeadershipTransferOnShutdown, persistent : true 01:50:02,104 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-1-shard-people-testLeadershipTransferOnShutdown, persistent : true 01:50:02,105 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 01:50:02,105 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:02,105 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:02,105 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-1-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 01:50:02,105 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:02,105 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:50:02,105 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-notifier#1184154446 created and ready for shard:member-1-shard-people-testLeadershipTransferOnShutdown 01:50:02,105 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#732818263] 01:50:02,105 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-1-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:02,105 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 01:50:02,105 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized 01:50:02,105 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:02,105 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:02,105 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-2018160395] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:02,105 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1353936402] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:02,105 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-notifier#1184154446] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:02,105 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#732818263] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:02,105 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#2135020546 created and ready for shard:member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:02,105 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 01:50:02,105 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is initialized 01:50:02,105 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#2135020546] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:02,105 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#598252140] 01:50:02,105 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#598252140] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:02,105 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#1682932222] 01:50:02,106 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-notifier#-1833564683 created and ready for shard:member-2-shard-people-testLeadershipTransferOnShutdown 01:50:02,106 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 01:50:02,106 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:50:02,106 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#1861598234] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:02,106 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-notifier#1164419373] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:02,106 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#1682932222] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:02,106 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from null to Follower 01:50:02,106 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:02,106 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:50:02,107 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:02,107 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 01:50:02,107 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:50:02,107 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from null to Follower 01:50:02,109 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:02,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:02,109 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:02,109 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:02,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:02,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:50:02,109 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:02,109 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:50:02,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 01:50:02,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-1-shard-cars-testLeadershipTransferOnShutdown is initialized 01:50:02,109 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 01:50:02,109 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 01:50:02,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 01:50:02,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-1-shard-people-testLeadershipTransferOnShutdown is initialized 01:50:02,109 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:50:02,109 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:50:02,110 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from null to Follower 01:50:02,110 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from null to Follower 01:50:02,115 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:02,116 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:02,152 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1662680583] 01:50:02,152 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 01:50:02,152 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-2018160395] 01:50:02,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:02,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 01:50:02,408 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:02,408 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 01:50:02,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 01:50:02,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 01:50:02,408 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-3-shard-cars-testLeadershipTransferOnShutdown] 01:50:02,408 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:02,408 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 01:50:02,408 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 01:50:02,408 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:02,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 01:50:02,408 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:02,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:02,408 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 01:50:02,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:50:02,408 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:02,408 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:50:02,408 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:50:02,408 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:50:02,408 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 01:50:02,408 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 01:50:02,657 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:50:02,657 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=2, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 01:50:02,657 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 2 01:50:02,687 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:50:02,687 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=2, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:02,687 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 2 01:50:02,928 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:50:02,928 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:02,928 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 3 01:50:02,967 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:50:02,967 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 01:50:02,967 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 3 01:50:03,38 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:03,38 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:03,38 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:03,38 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:03,38 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:03,38 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:50:03,38 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:03,38 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:03,38 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:03,38 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:03,38 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:50:03,38 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:03,38 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-3-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:03,38 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:03,38 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-3-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:50:03,38 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-2-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:50:03,44 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:03,44 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:03,44 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:03,44 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:50:03,44 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:50:03,44 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:03,44 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:03,44 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:03,44 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-3-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:03,44 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:50:03,44 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:50:03,44 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-3-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:50:03,87 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:03,177 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:50:03,177 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:03,177 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 4 01:50:03,177 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:03,177 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:03,179 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:03,179 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 01:50:03,179 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-2018160395] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,179 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:03,179 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:03,179 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 01:50:03,179 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:03,179 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:03,179 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:03,180 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:03,180 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:03,180 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-3-shard-cars-testLeadershipTransferOnShutdown] 01:50:03,180 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:03,180 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:03,180 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:03,180 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 01:50:03,181 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [] 01:50:03,181 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 01:50:03,181 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:50:03,181 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:03,181 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:03,181 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 01:50:03,181 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-2018160395] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,181 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 01:50:03,181 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:50:03,181 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:03,181 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:03,181 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:03,182 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done false 01:50:03,182 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 01:50:03,182 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:03,197 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 01:50:03,197 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 01:50:03,197 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 01:50:03,197 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 01:50:03,198 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 01:50:03,198 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=1, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-people-testLeadershipTransferOnShutdown 01:50:03,198 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=1, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-testLeadershipTransferOnShutdown 01:50:03,198 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:03,198 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 01:50:03,198 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:03,198 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:50:03,198 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:50:03,198 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 01:50:03,199 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVote [term=1, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1], my term is 3 01:50:03,199 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#2108128209] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,199 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): In requestVote: RequestVote [term=1, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: member-1-shard-people-testLeadershipTransferOnShutdown, lastIndex: -1, lastTerm: -1 01:50:03,199 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): requestVote returning: RequestVoteReply [term=3, voteGranted=false] 01:50:03,200 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=false], my term is 1 01:50:03,200 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Term 3 in "RequestVoteReply [term=3, voteGranted=false]" message is greater than Candidate's term 1 - switching to Follower 01:50:03,200 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=null 01:50:03,200 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Follower, election term: 3 01:50:03,200 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:03,200 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:50:03,200 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Follower 01:50:03,200 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Candidate to Follower 01:50:03,238 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:50:03,238 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 01:50:03,238 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 4 01:50:03,238 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testLeadershipTransferOnShutdown 01:50:03,238 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-testLeadershipTransferOnShutdown 01:50:03,238 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:03,239 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 01:50:03,239 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:03,239 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 3 - updating term 01:50:03,239 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 01:50:03,239 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1353936402] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,239 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:03,239 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 01:50:03,239 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:03,239 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:03,239 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:03,239 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 01:50:03,239 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:03,239 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:03,240 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:03,240 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 01:50:03,240 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: became leader with frontend state for [] 01:50:03,240 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 01:50:03,240 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:50:03,240 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:03,240 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:03,240 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Candidate to Leader 01:50:03,240 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:50:03,240 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown: Closing 01:50:03,240 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:03,240 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1353936402] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,241 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:50:03,241 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:03,241 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:03,241 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:50:03,241 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-people-testLeadershipTransferOnShutdown status sync done false 01:50:03,248 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 01:50:03,248 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:03,256 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1662680583] 01:50:03,257 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : people 01:50:03,257 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1353936402] 01:50:03,257 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:03,259 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:03,262 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 01:50:03,266 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1092663626] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,267 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 01:50:03,268 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 01:50:03,268 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:03,268 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7c18bda1 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6b7e2b67 01:50:03,268 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1301dc07 01:50:03,268 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1710165003268, None)) 01:50:03,268 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 01:50:03,269 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:03,269 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165003267,0,0)) 01:50:03,269 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-3-shard-people-testLeadershipTransferOnShutdown 01:50:03,269 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6b7e2b67 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@52057b31 01:50:03,269 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 01:50:03,270 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@40a258c5: creating a new transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:03,270 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:03,270 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:03,270 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:50:03,270 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:03,270 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:03,270 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:03,270 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:03,270 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:03,270 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:50:03,270 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@581f8348 01:50:03,270 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:03,270 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:03,270 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 01:50:03,270 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:03,270 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:03,270 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-3-shard-cars-testLeadershipTransferOnShutdown, persistent : true 01:50:03,270 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#458819312] 01:50:03,271 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} shard 1 01:50:03,271 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:03,271 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:03,271 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:03,271 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 01:50:03,271 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1172039374] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,271 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-3-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 01:50:03,271 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} 01:50:03,271 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:50:03,271 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:50:03,271 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#458819312] 01:50:03,271 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-notifier#-1411542417 created and ready for shard:member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:03,271 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} shard 2 01:50:03,271 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-notifier#-1411542417] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,271 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:03,271 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#1169066040] 01:50:03,271 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:50:03,271 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 01:50:03,271 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@40a258c5 readied transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:03,271 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@40a258c5: creating a new transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:50:03,271 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#1169066040] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,271 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:50:03,271 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:50:03,271 PM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:50:03,271 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@809938f, attempting to connect 01:50:03,271 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@40a258c5 readied transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:50:03,271 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:50:03,272 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown, primaryVersion=13] 01:50:03,272 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testLeadershipTransferOnShutdown 01:50:03,272 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1662680583] 01:50:03,273 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:03,273 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-people-testLeadershipTransferOnShutdown 01:50:03,273 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 01:50:03,273 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 01:50:03,273 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 01:50:03,274 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown, primaryVersion=13] 01:50:03,274 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-2018160395] 01:50:03,274 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:03,274 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:03,274 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1172039374] 01:50:03,274 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:03,274 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:03,274 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-2-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1172039374] 01:50:03,274 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:03,274 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:03,274 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-3-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1172039374] 01:50:03,274 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@46f17af4, attempting to connect 01:50:03,274 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-3-shard-people-testLeadershipTransferOnShutdown 01:50:03,274 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onShutDown 01:50:03,274 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Initiating leader transfer 01:50:03,274 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:03,274 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:50:03,275 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#2122666072] 01:50:03,275 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:50:03,275 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:50:03,275 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@583b389d 01:50:03,275 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-2-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#2122666072] 01:50:03,275 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@583b389d 01:50:03,275 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:50:03,275 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 01:50:03,275 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:50:03,275 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-3-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#2122666072] 01:50:03,275 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:03,275 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:03,275 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} is not yet known 01:50:03,275 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: not currently leader, rejecting request ConnectClientRequest{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testLeadershipTransferOnShutdown_member-1-shard-cars-testLeadershipTransferOnShutdown$a], minVersion=POTASSIUM, maxVersion=POTASSIUM}. isLeader: true, isLeaderActive: false,isLeadershipTransferInProgress: true. 01:50:03,275 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-3-shard-people-testLeadershipTransferOnShutdown, persistent : true 01:50:03,275 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505], modifications=2, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@69626138 01:50:03,275 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:50:03,275 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@58c9a0fa 01:50:03,275 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:50:03,275 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:03,275 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:03,276 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1c71bb06 skipping transactions [1] 01:50:03,276 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} scheduling timeout in 30 seconds 01:50:03,276 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-notifier#1016300974 created and ready for shard:member-3-shard-people-testLeadershipTransferOnShutdown 01:50:03,276 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#892951669] 01:50:03,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:03,276 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:03,276 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 01:50:03,276 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-3-shard-cars-testLeadershipTransferOnShutdown is initialized 01:50:03,276 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} is not yet known 01:50:03,276 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=8160, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:03,277 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-3-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 01:50:03,277 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown] 01:50:03,277 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done true 01:50:03,278 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:50:03,278 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:03,278 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:03,278 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#2122666072] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,279 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Connect attempt to cars failed, will retry org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-2018160395] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:03,279 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:50:03,279 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-2018160395] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:03,279 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-2018160395] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:03,280 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testLeadershipTransferOnShutdown, matchIndex: -1, lastIndex: -1, isVoting: true 01:50:03,280 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 01:50:03,280 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:03,280 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:03,280 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 01:50:03,280 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1353936402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:50:03,280 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1353936402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:03,280 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1c71bb06 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@58809da3 01:50:03,280 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} previous state FLUSHED 01:50:03,281 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@58809da3 01:50:03,281 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:50:03,281 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=OPEN} 01:50:03,281 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=OPEN} 01:50:03,281 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:50:03,281 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2219/0x00000008016a6000@20787c89 01:50:03,281 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1353936402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:50:03,281 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@38b7e6b5 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=749610877916} 01:50:03,281 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-628: slice: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505], modifications=1, protocol=THREE_PHASE}} 01:50:03,281 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-628: Message size: 389 bytes, total slices to send: 1 01:50:03,281 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-628: Message does not need to be sliced - sending original message 01:50:03,281 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@40a258c5 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1353936402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:03,281 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} 01:50:03,281 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@769b673c 01:50:03,281 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3422726 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@5547a208, splicing queue 01:50:03,281 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3422726 queue spliced 0 messages 01:50:03,281 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1353936402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 574.6 μs 01:50:03,281 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-2018160395] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:03,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 01:50:03,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 01:50:03,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:03,281 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 5 01:50:03,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:03,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:03,281 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:50:03,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 01:50:03,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:03,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:50:03,281 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 01:50:03,282 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:50:03,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testLeadershipTransferOnShutdown: allocating new open transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:50:03,282 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 01:50:03,282 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 01:50:03,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:50:03,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 ready 01:50:03,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: transitioned member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 to ready 01:50:03,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 initiating canCommit 01:50:03,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:50:03,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 validated 01:50:03,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:03,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 completed canCommit 01:50:03,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:03,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 01:50:03,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:03,282 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than leader's term 4 - switching to Follower 01:50:03,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 01:50:03,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 01:50:03,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:03,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:03,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:50:03,283 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 5 01:50:03,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-633: Closing 01:50:03,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:03,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:50:03,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown: Closing 01:50:03,283 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:03,283 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:03,283 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Leader to Follower 01:50:03,283 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:03,283 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:50:03,283 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 01:50:03,283 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Leader to Follower 01:50:03,283 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-1-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:03,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 01:50:03,283 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-3-shard-people-testLeadershipTransferOnShutdown is initialized 01:50:03,283 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-2-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:50:03,283 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 01:50:03,283 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:50:03,283 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-people-testLeadershipTransferOnShutdown from null to Follower 01:50:03,288 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0}} 01:50:03,288 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=749610877916, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0}} 01:50:03,288 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} canCommit completed 01:50:03,289 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:50:03,289 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:50:03,289 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 01:50:03,289 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:03,289 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:03,289 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:50:03,289 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 01:50:03,289 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [] 01:50:03,289 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 01:50:03,289 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:50:03,289 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:03,289 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 01:50:03,289 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:50:03,290 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown: Closing 01:50:03,290 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:03,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:03,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:50:03,290 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 01:50:03,290 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-2-shard-cars-testLeadershipTransferOnShutdown in 15.31 ms 01:50:03,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 01:50:03,290 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 01:50:03,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 01:50:03,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:03,290 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-1-shard-cars-testLeadershipTransferOnShutdown status sync done false 01:50:03,290 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-2018160395] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#2135020546] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#2135020546] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,290 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 01:50:03,290 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:03,290 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:03,290 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 01:50:03,291 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:03,291 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:50:03,291 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 01:50:03,291 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:03,291 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 4 - updating term 01:50:03,291 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 01:50:03,291 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:03,291 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:03,291 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:50:03,291 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 01:50:03,291 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:03,291 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntriesReply] from Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1172039374] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-2018160395] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-2018160395] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,292 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntriesReply] from Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1172039374] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-2018160395] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-2018160395] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,292 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:03,292 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:03,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown] 01:50:03,292 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:50:03,292 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done false 01:50:03,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 01:50:03,292 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:50:03,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:03,292 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:03,292 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:03,292 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from null to Follower 01:50:03,292 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:50:03,292 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:03,292 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:03,292 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done true 01:50:03,292 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done false 01:50:03,292 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:50:03,292 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:03,292 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:03,295 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 01:50:03,295 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:03,358 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-people-testLeadershipTransferOnShutdown status sync done true 01:50:03,358 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 01:50:03,358 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 01:50:03,359 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-3-shard-people-testLeadershipTransferOnShutdown: Closing 01:50:03,359 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:03,359 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:50:03,359 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:03,359 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:03,359 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:50:03,359 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 01:50:03,359 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:03,360 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-people-testLeadershipTransferOnShutdown status sync done false 01:50:03,408 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done true 01:50:03,479 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-people-testLeadershipTransferOnShutdown status sync done true 01:50:04,59 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:50:04,298 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 01:50:04,298 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:04,298 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:04,298 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#458819312] 01:50:04,298 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:04,298 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#655979141, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6f2843af] 01:50:04,298 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#655979141, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6f2843af] 01:50:04,299 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@4b1a003a, attempting to connect 01:50:04,299 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} is not yet known 01:50:04,299 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=13920, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:04,299 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:50:04,299 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#655979141], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:50:04,299 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#655979141], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:04,299 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@60a205d1 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@545f4ab9 01:50:04,300 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:50:04,300 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} previous state SEALED 01:50:04,300 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@545f4ab9 01:50:04,300 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:50:04,300 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 01:50:04,300 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505], modifications=2, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 01:50:04,300 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505], modifications=2, protocol=THREE_PHASE} 01:50:04,300 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} allocated sequence 0 01:50:04,300 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@69626138 01:50:04,300 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#655979141], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:50:04,300 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7d2c162b transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505], coordinated=true}, enqueuedTicks=749610522170} 01:50:04,301 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@545f4ab9 01:50:04,301 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:50:04,301 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} 01:50:04,301 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} 01:50:04,301 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testLeadershipTransferOnShutdown: allocating new ready transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:50:04,301 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505], modifications=1} 01:50:04,301 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 initiating canCommit 01:50:04,301 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:50:04,301 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 validated 01:50:04,301 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:04,301 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 completed canCommit 01:50:04,301 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@40a258c5 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#655979141], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:04,301 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 01:50:04,301 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} 01:50:04,301 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@1b435cc9 01:50:04,301 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6439cc4 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3414a5e7, splicing queue 01:50:04,301 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6439cc4 queue spliced 0 messages 01:50:04,301 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#655979141], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 1.691 ms 01:50:04,301 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505], coordinated=true} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 01:50:04,301 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505], coordinated=true}, enqueuedTicks=749610522170, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 01:50:04,301 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 01:50:04,301 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:50:04,301 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@232069fc 01:50:04,302 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 01:50:04,302 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} 01:50:04,302 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} allocated sequence 1 01:50:04,302 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@232069fc 01:50:04,302 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#655979141], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:04,302 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7d2c162b transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1197434505]}, enqueuedTicks=750636938672} 01:50:04,302 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 01:50:04,302 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@59b1b701 01:50:04,302 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} 01:50:04,302 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} allocated sequence 1 01:50:04,302 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1242/0x0000000801303c28@4cea02bd 01:50:04,302 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1353936402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:04,302 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@38b7e6b5 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1197434505]}, enqueuedTicks=750637183806} 01:50:04,302 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 initiating preCommit 01:50:04,302 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:50:04,302 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 candidate ready 01:50:04,302 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1: canCommit - candidate rootPath: / 01:50:04,303 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1: canCommit - messages: [] 01:50:04,303 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 01:50:04,303 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 prepared 01:50:04,303 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:04,303 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 completed preCommit 01:50:04,303 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1}} 01:50:04,303 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1197434505]}, enqueuedTicks=750636938672, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1}} 01:50:04,303 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:50:04,303 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 initiating preCommit 01:50:04,303 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:50:04,303 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 candidate ready 01:50:04,303 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2: canCommit - candidate rootPath: / 01:50:04,303 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2: canCommit - messages: [] 01:50:04,303 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2: preCommit - successfulFromPrevious: [] 01:50:04,303 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 prepared 01:50:04,303 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:04,303 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 completed preCommit 01:50:04,312 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1}} 01:50:04,312 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1197434505]}, enqueuedTicks=750637183806, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1}} 01:50:04,312 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:50:04,312 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:50:04,312 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@1355d542 01:50:04,312 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 01:50:04,312 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} 01:50:04,312 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} allocated sequence 2 01:50:04,312 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@1355d542 01:50:04,312 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#655979141], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:04,312 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7d2c162b transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1197434505]}, enqueuedTicks=750647473204} 01:50:04,312 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 2 01:50:04,312 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@47887196 01:50:04,312 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} 01:50:04,312 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} allocated sequence 2 01:50:04,312 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@47887196 01:50:04,312 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1353936402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:04,312 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 initiating commit 01:50:04,312 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@38b7e6b5 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1197434505]}, enqueuedTicks=750647667037} 01:50:04,312 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:50:04,313 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 170 01:50:04,313 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 01:50:04,313 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 01:50:04,313 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:04,313 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:04,313 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:04,313 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}]]] 01:50:04,313 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:04,313 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 submitted to persistence 01:50:04,314 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 01:50:04,314 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}]]] 01:50:04,314 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:50:04,314 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:04,314 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=170} 01:50:04,314 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 01:50:04,314 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 1 01:50:04,314 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:04,315 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 01:50:04,315 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 initiating commit 01:50:04,315 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:50:04,315 PM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 174 01:50:04,315 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:50:04,315 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:50:04,315 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:04,315 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:04,315 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 01:50:04,315 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 01:50:04,315 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:04,315 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 01:50:04,315 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 01:50:04,315 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 submitted to persistence 01:50:04,316 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:50:04,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 01:50:04,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:04,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 0 01:50:04,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:50:04,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=170} 01:50:04,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:50:04,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@6a0b3ab9}, purged=MutableUnsignedLongSet{size=0}} 01:50:04,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 01:50:04,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:50:04,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1: commit - successfulFromPrevious: [] 01:50:04,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:50:04,316 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2}} 01:50:04,316 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1197434505]}, enqueuedTicks=750647473204, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2}} 01:50:04,317 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 01:50:04,317 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@60a205d1 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 01:50:04,317 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-190092899] with name member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 01:50:04,317 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 01:50:04,317 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 01:50:04,317 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 01:50:04,317 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@7f745ab9 01:50:04,317 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:50:04,317 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:04,317 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 01:50:04,317 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=174} 01:50:04,317 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:50:04,317 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} 01:50:04,317 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED}: initiating purge 01:50:04,317 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=DONE} allocated sequence 3 01:50:04,317 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@7e50f48a 01:50:04,317 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-190092899] 01:50:04,317 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 1 01:50:04,317 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#655979141], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:04,317 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:04,317 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7d2c162b transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1197434505]}, enqueuedTicks=750651808426} 01:50:04,317 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:04,317 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:50:04,317 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:50:04,317 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-190092899] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:04,317 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:50:04,317 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 01:50:04,317 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 01:50:04,317 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 106.6 μs 01:50:04,317 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:04,317 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 01:50:04,317 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:50:04,317 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:04,317 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=174} 01:50:04,317 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:50:04,317 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 1 01:50:04,317 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:04,317 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:50:04,318 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:50:04,318 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 01:50:04,318 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 01:50:04,318 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 01:50:04,318 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 01:50:04,318 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:04,318 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Setting last applied to 0 01:50:04,318 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:50:04,318 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=174} 01:50:04,318 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:50:04,319 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-1-shard-people-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@58e762c2}, purged=MutableUnsignedLongSet{size=0}} 01:50:04,319 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 01:50:04,319 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:50:04,319 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2: commit - successfulFromPrevious: [] 01:50:04,319 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 01:50:04,319 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#740761285] with name member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 01:50:04,319 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:04,319 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:50:04,319 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 01:50:04,319 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:04,319 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#740761285] 01:50:04,319 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:50:04,319 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 01:50:04,319 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:04,319 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 01:50:04,319 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:50:04,319 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=170} 01:50:04,319 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 62.78 μs 01:50:04,319 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:50:04,320 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-3-shard-cars-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@6e7e9100}, purged=MutableUnsignedLongSet{size=0}} 01:50:04,320 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 01:50:04,320 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 01:50:04,320 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:04,320 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:50:04,320 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 01:50:04,320 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:50:04,320 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=174} 01:50:04,320 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-3-shard-people-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:50:04,320 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1274911861] with name member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 01:50:04,320 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1274911861] 01:50:04,320 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:50:04,320 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:50:04,320 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-3-shard-people-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@8d1afaa}, purged=MutableUnsignedLongSet{size=0}} 01:50:04,320 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 01:50:04,320 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-people-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:50:04,320 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-29508851] with name member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 01:50:04,320 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 82.16 μs 01:50:04,320 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:50:04,320 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-29508851] 01:50:04,320 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:50:04,320 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 50.95 μs 01:50:04,321 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:04,321 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=5, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}]]] 01:50:04,321 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 01:50:04,321 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:04,321 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 01:50:04,321 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:50:04,321 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=174} 01:50:04,321 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-people-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:50:04,321 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2}} 01:50:04,321 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendMetadata - member-2-shard-people-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@2febb029}, purged=MutableUnsignedLongSet{size=0}} 01:50:04,321 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 01:50:04,321 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1197434505]}, enqueuedTicks=750647667037, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2}} 01:50:04,321 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:50:04,322 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} doCommit completed 01:50:04,322 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1609488558] with name member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 01:50:04,322 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1c71bb06 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} 01:50:04,322 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR}: initiating purge 01:50:04,322 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 3 01:50:04,322 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1609488558] 01:50:04,322 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@1216782d 01:50:04,322 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:50:04,322 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} 01:50:04,322 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:50:04,322 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED}: initiating purge 01:50:04,322 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=DONE} allocated sequence 3 01:50:04,322 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@1aaaff61 01:50:04,322 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1609488558] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:04,322 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1353936402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:04,322 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@38b7e6b5 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1197434505]}, enqueuedTicks=750656821759} 01:50:04,322 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 126.5 μs 01:50:04,323 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=FLUSHED} allocated sequence 0 01:50:04,323 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@3939521c 01:50:04,323 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7d2c162b transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505], coordinated=false}, enqueuedTicks=750658154120} 01:50:04,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testLeadershipTransferOnShutdown: allocating new ready transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:50:04,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 initiating direct canCommit 01:50:04,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:50:04,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 validated 01:50:04,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:04,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 initiating direct preCommit 01:50:04,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:50:04,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 candidate ready 01:50:04,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1: canCommit - candidate rootPath: / 01:50:04,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1: canCommit - messages: [] 01:50:04,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 01:50:04,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 prepared 01:50:04,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:04,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 initiating direct commit 01:50:04,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:50:04,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 229 01:50:04,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 01:50:04,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 01:50:04,324 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:04,324 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:04,324 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:04,324 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:04,324 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 submitted to persistence 01:50:04,324 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:50:04,324 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 01:50:04,324 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 01:50:04,324 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:04,324 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 01:50:04,324 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=5, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}]]] 01:50:04,324 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:50:04,324 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:04,324 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62} 01:50:04,324 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 01:50:04,324 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 2 01:50:04,324 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=1, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:04,324 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 01:50:04,324 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 01:50:04,325 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:04,325 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 1 01:50:04,325 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:50:04,325 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62} 01:50:04,325 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: replication of member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@3ee845ff 01:50:04,325 PM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testLeadershipTransferOnShutdown: finished purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:50:04,325 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 01:50:04,325 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:50:04,325 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3}} 01:50:04,325 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:04,325 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:04,325 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1197434505]}, enqueuedTicks=750651808426, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3}} 01:50:04,325 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=DONE}: purge completed 01:50:04,325 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@545f4ab9 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=DONE} 01:50:04,325 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=1, prevLogTerm=5, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}]]] 01:50:04,325 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:50:04,325 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@60a205d1 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 01:50:04,325 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 01:50:04,326 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 01:50:04,326 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=1, prevLogTerm=5, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}]]] 01:50:04,326 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:50:04,326 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:50:04,326 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=229} 01:50:04,326 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 01:50:04,326 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 3 01:50:04,326 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 1 01:50:04,326 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:04,326 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 2 01:50:04,326 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 01:50:04,326 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:50:04,326 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62} 01:50:04,326 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 01:50:04,326 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:50:04,326 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 01:50:04,326 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 01:50:04,327 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 01:50:04,327 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:50:04,327 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 2 01:50:04,327 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 01:50:04,327 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=229} 01:50:04,327 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:50:04,327 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 01:50:04,327 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:50:04,327 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1: commit - successfulFromPrevious: [] 01:50:04,327 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 01:50:04,327 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:50:04,327 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505], coordinated=false} with Envelope{sessionId=1, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0}} 01:50:04,327 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505], coordinated=false}, enqueuedTicks=750658154120, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0}} 01:50:04,327 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=FLUSHED} directCommit completed 01:50:04,327 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=FLUSHED}: initiating purge 01:50:04,327 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=DONE} allocated sequence 1 01:50:04,327 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 49.32 μs 01:50:04,327 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:04,327 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@6a65f3b0 01:50:04,327 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#655979141], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:04,327 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7d2c162b transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1197434505]}, enqueuedTicks=750662111886} 01:50:04,327 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 01:50:04,327 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:50:04,327 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 01:50:04,327 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 01:50:04,327 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:04,327 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 01:50:04,327 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 2 01:50:04,327 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:50:04,327 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 2 01:50:04,327 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 01:50:04,327 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=229} 01:50:04,328 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:50:04,328 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 01:50:04,328 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:50:04,328 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 01:50:04,328 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:50:04,328 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 47.44 μs 01:50:04,328 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 3 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:04,328 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}]]] 01:50:04,329 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}]]] 01:50:04,329 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:50:04,329 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:50:04,329 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63} 01:50:04,329 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 01:50:04,329 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 4 01:50:04,329 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=3, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:04,329 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@40a258c5: creating a new snapshot member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 01:50:04,329 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 01:50:04,329 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 01:50:04,329 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2cf2c041: creating a new snapshot member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:04,329 PM [main] [DEBUG] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 01:50:04,329 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:04,329 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:04,329 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1092663626] 01:50:04,329 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:04,329 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} shard 1 01:50:04,329 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 01:50:04,329 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:04,329 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:04,329 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 01:50:04,329 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1657302664], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@7c618f8f 01:50:04,329 PM [main] [DEBUG] AbstractClientConnection - member-3-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:50:04,329 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@3b1cca47, attempting to connect 01:50:04,330 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 3, nextIndex: 4 01:50:04,330 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 01:50:04,330 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 3 01:50:04,330 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 2 to 3 01:50:04,330 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63} 01:50:04,330 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: replication of member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@69bf2a9 01:50:04,330 PM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testLeadershipTransferOnShutdown: finished purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:50:04,330 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 01:50:04,330 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:50:04,330 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1}} 01:50:04,330 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1197434505]}, enqueuedTicks=750662111886, sessionId=1, txSequence=5} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1}} 01:50:04,330 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=DONE}: purge completed 01:50:04,330 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:50:04,330 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@545f4ab9 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=DONE} 01:50:04,330 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=3 01:50:04,331 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} is not yet known 01:50:04,331 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=6150, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:04,331 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:50:04,332 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#655979141], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:50:04,332 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#655979141], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:04,332 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@22198047 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7f8c1828 01:50:04,332 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:50:04,332 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7f8c1828 01:50:04,332 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:50:04,332 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 01:50:04,332 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1657302664], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 01:50:04,332 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:04,332 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1657302664], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1248/0x0000000801301d10@305822f0 01:50:04,332 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-3-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#655979141], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:50:04,332 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7b5a8876 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1657302664], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=750664671867} 01:50:04,332 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2cf2c041 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#655979141], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:04,332 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 01:50:04,332 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@6488926d 01:50:04,332 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2b02dacf superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@5a3c32fd, splicing queue 01:50:04,332 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2b02dacf queue spliced 0 messages 01:50:04,332 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#655979141], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 544.2 μs 01:50:04,332 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 3 01:50:04,332 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 01:50:04,332 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 3 01:50:04,332 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 2 to 3 01:50:04,332 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63} 01:50:04,332 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 01:50:04,332 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:50:04,332 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=3 01:50:04,333 PM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testLeadershipTransferOnShutdown: allocating new open snapshot member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:50:04,333 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageSlicer - member-2-shard-cars-testLeadershipTransferOnShutdown_slicer-id-630: slice: identifier: member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 01:50:04,334 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SlicedMessageState - member-2-shard-cars-testLeadershipTransferOnShutdown_slicer-id-630: Message size: 363 bytes, total slices to send: 1 01:50:04,334 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageSlicer - member-2-shard-cars-testLeadershipTransferOnShutdown_slicer-id-630: Message does not need to be sliced - sending original message 01:50:04,335 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1657302664], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 01:50:04,335 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1657302664], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=750664671867, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 01:50:04,335 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0} 01:50:04,335 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:04,335 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:04,335 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-3-shard-people-testLeadershipTransferOnShutdown 01:50:04,335 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 01:50:04,335 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onShutDown 01:50:04,335 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onShutDown 01:50:04,335 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:04,335 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 01:50:04,335 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-3-shard-people-testLeadershipTransferOnShutdown 01:50:04,335 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 01:50:04,335 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:50:04,335 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:50:04,336 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 01:50:04,336 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 01:50:04,336 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:50:04,336 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:04,336 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 01:50:04,336 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-3-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:50:04,337 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:04,337 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-1-shard-people-testLeadershipTransferOnShutdown 01:50:04,337 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 01:50:04,337 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onShutDown 01:50:04,337 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Initiating leader transfer 01:50:04,337 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7bf9ef04 01:50:04,337 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7bf9ef04 01:50:04,337 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 01:50:04,337 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:04,337 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 01:50:04,337 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:50:04,337 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 01:50:04,337 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:04,337 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:04,337 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:04,337 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 01:50:04,337 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 01:50:04,338 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 01:50:04,338 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:50:04,338 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:04,338 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62} 01:50:04,338 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 01:50:04,338 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 2 01:50:04,338 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testLeadershipTransferOnShutdown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:04,339 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 01:50:04,339 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown] 01:50:04,339 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:50:04,339 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:04,339 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 01:50:04,339 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 01:50:04,339 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:04,339 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@38b7e6b5 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@4a048020, splicing queue 01:50:04,339 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Setting last applied to 1 01:50:04,339 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:50:04,339 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62} 01:50:04,339 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1353936402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:50:04,339 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: replication of member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@433c006c 01:50:04,339 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@38b7e6b5 queue spliced 1 messages 01:50:04,339 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1353936402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1353936402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:04,339 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testLeadershipTransferOnShutdown: finished purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:50:04,339 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: refreshing backend for shard 2 01:50:04,339 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 01:50:04,339 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1353936402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:50:04,339 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:50:04,339 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:50:04,339 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:50:04,339 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 01:50:04,339 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#458819312] 01:50:04,339 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:04,339 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testLeadershipTransferOnShutdown, matchIndex: 1, lastIndex: 1, isVoting: true 01:50:04,339 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 01:50:04,339 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:04,339 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:04,339 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 01:50:04,339 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 01:50:04,339 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:50:04,339 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 01:50:04,339 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Scheduling 2000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:50:04,339 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 01:50:04,340 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 1 01:50:04,340 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:04,340 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 01:50:04,340 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:50:04,340 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62} 01:50:04,340 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 01:50:04,340 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:50:04,340 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 01:50:04,340 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 01:50:04,340 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3}} not found in inflight queue, checking pending queue 01:50:04,340 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 01:50:04,340 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 01:50:04,340 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3}} 01:50:04,340 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 5 01:50:04,340 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1197434505]}, enqueuedTicks=750656821759, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3}} 01:50:04,340 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=DONE}: purge completed 01:50:04,340 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@58809da3 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=DONE} 01:50:04,340 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR}: purge completed 01:50:04,340 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-people-testLeadershipTransferOnShutdown 01:50:04,340 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1c71bb06 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} 01:50:04,340 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] to peer member-3-shard-people-testLeadershipTransferOnShutdown 01:50:04,341 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:50:04,341 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 01:50:04,341 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:04,341 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:50:04,341 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown: Closing 01:50:04,341 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:50:04,341 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 01:50:04,341 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 01:50:04,341 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 01:50:04,341 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 01:50:04,341 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 01:50:04,341 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:04,341 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 01:50:04,341 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:04,341 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4]" message is greater than leader's term 4 - switching to Follower 01:50:04,341 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 01:50:04,341 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 01:50:04,341 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 01:50:04,341 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 01:50:04,341 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:50:04,342 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 5 01:50:04,342 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-people-testLeadershipTransferOnShutdown (Leader)_slicer-id-634: Closing 01:50:04,342 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:04,342 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:50:04,342 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown: Closing 01:50:04,342 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Leader to Follower 01:50:04,342 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 01:50:04,342 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Leader to Follower 01:50:04,342 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:50:04,342 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:50:04,342 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 01:50:04,342 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:04,342 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:04,342 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:50:04,342 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 01:50:04,343 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: became leader with frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 01:50:04,343 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 01:50:04,343 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:50:04,343 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 01:50:04,343 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:04,343 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Candidate to Leader 01:50:04,343 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 01:50:04,343 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is ready - notifying 1 OnShardInitialized callbacks 01:50:04,343 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for people: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#2108128209, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@76fc5862] 01:50:04,343 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:50:04,343 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#2108128209, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@76fc5862] 01:50:04,343 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@50e387f0, attempting to connect 01:50:04,343 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown: Closing 01:50:04,343 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:50:04,343 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-2-shard-people-testLeadershipTransferOnShutdown in 6.532 ms 01:50:04,343 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 01:50:04,343 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-1-shard-people-testLeadershipTransferOnShutdown status sync done false 01:50:04,344 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-1-shard-people-testLeadershipTransferOnShutdown 01:50:04,344 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 01:50:04,344 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:50:04,344 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 01:50:04,344 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:50:04,344 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#2108128209], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present} 01:50:04,344 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 01:50:04,344 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1353936402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#2108128209], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 01:50:04,344 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@58809da3 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@7b367a2 01:50:04,344 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:50:04,344 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@40a258c5 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#2108128209], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 01:50:04,344 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@6958f2f4 01:50:04,344 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:04,344 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@503080ac superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@4f8637fe, splicing queue 01:50:04,344 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@503080ac queue spliced 0 messages 01:50:04,344 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 01:50:04,344 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:50:04,344 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1353936402], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#2108128209], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} in 191.6 μs 01:50:04,344 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:50:04,345 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:04,345 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:04,345 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 01:50:04,345 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:04,349 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:04,349 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:04,349 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:04,350 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:04,360 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:50:04,360 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - Stopping Shard member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:04,360 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageSlicer - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-638: Closing 01:50:04,360 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-2-shard-people-testLeadershipTransferOnShutdown 01:50:04,360 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:50:04,360 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageSlicer - member-2-shard-people-testLeadershipTransferOnShutdown (Leader)_slicer-id-639: Closing 01:50:04,360 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:50:04,360 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 01:50:04,361 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:04,361 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:05,368 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-8777707846164332390] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:05,378 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [735413509306248000] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:05,380 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:05,380 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:05,381 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:05.382] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:05,388 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:05,388 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:05,390 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:05.391] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:05,404 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:05,409 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3354945518332090119] 01:50:05,409 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:05,410 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:05,410 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:05,416 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:05,416 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:05,421 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:05,421 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:05,421 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:05,425 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:05,429 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1936055271537579471] 01:50:05,429 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:05,431 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:05,431 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:05,436 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:05,436 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:05,441 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:05,454 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-6067031491400745464] 01:50:05,454 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:05,456 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:05,456 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:05,468 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:05,469 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:05,470 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionForwardedToLeaderAfterRetry 01:50:05,470 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:05,470 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:05,474 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:05,476 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:05,484 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1810542820] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:05,486 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 01:50:05,499 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionForwardedToLeaderAfterRetry 01:50:05,499 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: no initial generation override, starting from 0 01:50:05,499 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:50:05,499 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:05,499 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6def6166 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@561672a4 01:50:05,499 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4f2dd235 01:50:05,499 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:05,499 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:05,499 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:05,499 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:05,499 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, 0, 1710165005499, None)) 01:50:05,500 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165005498,0,0)) 01:50:05,500 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@561672a4 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@52b28239 01:50:05,500 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@33b24295 01:50:05,500 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} started 01:50:05,500 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:05,500 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:05,500 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:05,500 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:05,500 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:05,500 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428] 01:50:05,500 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:05,501 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, persistent : true 01:50:05,501 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 01:50:05,501 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, persistent : true 01:50:05,501 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 01:50:05,501 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:05,501 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:05,502 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-1832043142 created and ready for shard:member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:05,502 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:05,505 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionForwardedToLeaderAfterRetry 01:50:05,505 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:05,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:05,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948] 01:50:05,507 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#1259159274] 01:50:05,507 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-1832043142] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:05,507 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#899454273] 01:50:05,507 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#1259159274] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:05,507 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#899454273] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:05,511 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:05,511 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:05,511 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:05,511 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:05,511 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [cars] 01:50:05,511 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:50:05,511 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is initialized 01:50:05,511 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:05,511 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:50:05,511 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [people] 01:50:05,511 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:05,511 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is initialized 01:50:05,512 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 01:50:05,512 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 01:50:05,512 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-1904233577 created and ready for shard:member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:05,512 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 01:50:05,512 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 01:50:05,512 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from null to Follower 01:50:05,512 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-1904233577] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:05,512 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from null to Follower 01:50:05,513 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 01:50:05,513 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-565591775] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:05,516 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: no initial generation override, starting from 0 01:50:05,516 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionForwardedToLeaderAfterRetry 01:50:05,516 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:50:05,516 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:05,516 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@756c3f92 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@9809a56 01:50:05,516 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@158c828b 01:50:05,516 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:05,516 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:05,516 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, 0, 1710165005516, None)) 01:50:05,516 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:05,516 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:05,516 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165005515,0,0)) 01:50:05,516 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@9809a56 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4912ac65 01:50:05,516 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@59ecb9a5 01:50:05,516 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} started 01:50:05,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:05,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:05,517 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:05,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:05,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114] 01:50:05,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:05,517 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, persistent : true 01:50:05,517 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:05,517 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:05,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:05,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392] 01:50:05,517 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#262351151 created and ready for shard:member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:05,517 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 01:50:05,517 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#262351151] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:05,517 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, persistent : true 01:50:05,517 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:05,517 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#130130595] 01:50:05,518 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#1498159656 created and ready for shard:member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:05,518 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#130130595] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:05,518 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:05,518 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:05,518 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:50:05,518 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:05,518 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:05,518 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [cars] 01:50:05,518 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is initialized 01:50:05,518 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 01:50:05,518 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 01:50:05,518 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from null to Follower 01:50:05,521 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:05,523 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#1699084773] 01:50:05,523 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 01:50:05,524 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:05,524 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:05,524 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:05,524 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [people] 01:50:05,524 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is initialized 01:50:05,524 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:50:05,524 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:05,524 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:05,525 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#1498159656] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:05,525 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 01:50:05,525 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#1699084773] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:05,525 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 01:50:05,525 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from null to Follower 01:50:05,567 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1810542820] 01:50:05,567 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:05,567 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428] 01:50:05,741 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:05,741 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Received ElectionTimeout - switching to Candidate 01:50:05,741 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 01:50:05,741 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=1, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:05,741 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 1 01:50:05,741 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:05,741 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:50:05,741 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:50:05,741 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:05,742 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 01:50:05,742 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 01:50:05,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:05,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Received ElectionTimeout - switching to Candidate 01:50:05,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 01:50:05,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:05,782 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 1 01:50:05,782 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:05,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:50:05,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:50:05,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:05,782 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 01:50:05,782 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 01:50:05,982 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:50:05,982 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=2, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:05,982 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 2 01:50:06,21 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:50:06,21 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,21 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 2 01:50:06,124 PM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:06,124 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:06,261 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:50:06,261 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,261 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 3 01:50:06,292 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:50:06,292 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:06,292 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 3 01:50:06,433 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:06,433 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:06,433 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:06,433 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,433 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,433 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:06,433 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:06,433 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:06,433 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,433 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,433 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:06,433 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,433 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:06,433 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:06,439 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:06,439 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,439 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,439 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:06,439 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:06,439 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:06,439 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,439 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,439 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:06,439 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:06,481 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:06,571 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:50:06,571 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,571 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 4 01:50:06,571 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,572 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:06,572 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 01:50:06,572 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:06,572 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:06,572 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=null 01:50:06,573 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:06,573 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,573 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:06,574 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:06,574 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:06,574 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 01:50:06,574 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:06,574 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:06,574 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 01:50:06,574 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [] 01:50:06,574 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:50:06,574 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:06,574 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:06,574 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 01:50:06,574 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 01:50:06,575 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:06,575 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,575 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done false 01:50:06,575 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:50:06,575 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:06,575 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:06,577 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: -1, nextIndex: 0 01:50:06,577 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:06,601 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:50:06,601 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:06,601 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 4 01:50:06,601 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:06,602 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:06,602 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 01:50:06,602 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:06,602 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:06,602 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=null 01:50:06,602 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:06,602 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:06,602 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:06,603 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:06,603 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:06,603 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 01:50:06,603 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:06,603 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:06,603 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 01:50:06,603 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [] 01:50:06,603 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 01:50:06,603 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:50:06,603 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:06,603 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:06,603 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 01:50:06,603 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 01:50:06,604 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:06,604 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-1-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:06,604 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done false 01:50:06,604 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:50:06,604 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:06,604 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:06,604 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 01:50:06,605 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: -1, nextIndex: 0 01:50:06,605 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:06,672 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1810542820] 01:50:06,672 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 01:50:06,672 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948] 01:50:06,672 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1810542820] 01:50:06,672 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:06,672 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428] 01:50:06,673 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428] 01:50:06,673 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-565591775] 01:50:06,673 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:06,673 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114] 01:50:06,673 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114] 01:50:06,673 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@347bbdd6: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:06,673 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 01:50:06,673 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:06,673 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:06,673 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-565591775] 01:50:06,673 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} shard 1 01:50:06,673 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:06,673 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:06,674 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 01:50:06,674 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:50:06,674 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:50:06,674 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-565591775] 01:50:06,674 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} shard 2 01:50:06,674 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:06,674 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:50:06,674 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@347bbdd6 readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:06,674 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@9ce3b49 01:50:06,674 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:50:06,674 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@3f73bb46 01:50:06,674 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} scheduling timeout in 30 seconds 01:50:06,676 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:06,676 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:50:06,676 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:50:06,676 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:50:06,676 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:50:06,676 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:50:06,676 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@12858b0e, attempting to connect 01:50:06,676 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@50f7708b, attempting to connect 01:50:06,678 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 01:50:06,678 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=12130, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:50:06,678 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 01:50:06,678 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=12550, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:50:06,679 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:50:06,679 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:50:06,679 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:50:06,679 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:06,679 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2b2bc43b instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@497c91ad 01:50:06,679 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:50:06,679 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@497c91ad 01:50:06,679 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:50:06,679 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} 01:50:06,680 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} 01:50:06,680 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:06,680 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2219/0x00000008016a6000@68c857a6 01:50:06,680 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:50:06,680 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3d381b96 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=753009258889} 01:50:06,680 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE}} 01:50:06,680 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: Message size: 396 bytes, total slices to send: 1 01:50:06,680 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: Message does not need to be sliced - sending original message 01:50:06,680 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@347bbdd6 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:06,680 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 01:50:06,680 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@59973356 01:50:06,680 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5f0ef6bb superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@520a019a, splicing queue 01:50:06,680 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5f0ef6bb queue spliced 0 messages 01:50:06,680 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.088 ms 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@19d000a9 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@21e17117 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} previous state FLUSHED 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@21e17117 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2219/0x00000008016a6000@b7626cb 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2de36ba9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=753009369841} 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE}} 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: Message size: 401 bytes, total slices to send: 1 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: Message does not need to be sliced - sending original message 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@347bbdd6 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@7f911e50 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3288d046 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@446ac58b, splicing queue 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3288d046 queue spliced 0 messages 01:50:06,681 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 828.0 μs 01:50:06,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:50:06,685 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:50:06,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:50:06,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 ready 01:50:06,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transitioned member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 to ready 01:50:06,686 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 initiating canCommit 01:50:06,686 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:50:06,686 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:50:06,686 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 validated 01:50:06,686 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:06,686 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 ready 01:50:06,686 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transitioned member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 to ready 01:50:06,686 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 completed canCommit 01:50:06,686 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 initiating canCommit 01:50:06,686 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:50:06,686 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 validated 01:50:06,686 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:06,686 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 completed canCommit 01:50:06,688 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 01:50:06,688 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=753009258889, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 01:50:06,688 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 01:50:06,688 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 01:50:06,688 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=753009369841, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 01:50:06,688 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} canCommit completed 01:50:06,688 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:50:06,688 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@3229028f 01:50:06,688 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 01:50:06,688 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} allocated sequence 1 01:50:06,688 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1242/0x0000000801303c28@4a4563f0 01:50:06,688 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:06,688 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3d381b96 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=753023616466} 01:50:06,688 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 01:50:06,688 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@684e4875 01:50:06,688 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 01:50:06,688 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} allocated sequence 1 01:50:06,688 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1242/0x0000000801303c28@3724095c 01:50:06,688 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:06,688 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2de36ba9 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=753023748638} 01:50:06,689 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 initiating preCommit 01:50:06,689 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 initiating preCommit 01:50:06,689 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:50:06,689 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:50:06,689 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 candidate ready 01:50:06,689 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 candidate ready 01:50:06,689 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2: canCommit - candidate rootPath: / 01:50:06,689 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1: canCommit - candidate rootPath: / 01:50:06,689 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2: canCommit - messages: [] 01:50:06,689 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1: canCommit - messages: [] 01:50:06,689 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2: preCommit - successfulFromPrevious: [] 01:50:06,689 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 01:50:06,689 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 prepared 01:50:06,689 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 prepared 01:50:06,689 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:06,689 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:06,689 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 completed preCommit 01:50:06,689 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 completed preCommit 01:50:06,690 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1}} 01:50:06,690 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=753023748638, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1}} 01:50:06,690 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:50:06,690 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1}} 01:50:06,690 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=753023616466, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1}} 01:50:06,690 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:50:06,690 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:50:06,690 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@2c58f8f2 01:50:06,690 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 01:50:06,690 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} allocated sequence 2 01:50:06,690 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@2c58f8f2 01:50:06,690 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:06,690 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3d381b96 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=753025324254} 01:50:06,690 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 2 01:50:06,690 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@36cf00ec 01:50:06,690 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 01:50:06,690 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} allocated sequence 2 01:50:06,690 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@36cf00ec 01:50:06,690 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:06,690 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2de36ba9 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=753025442136} 01:50:06,691 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 initiating commit 01:50:06,691 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:50:06,691 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 initiating commit 01:50:06,691 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:50:06,691 PM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 180 01:50:06,691 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 184 01:50:06,691 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:50:06,691 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 01:50:06,691 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:50:06,691 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 01:50:06,691 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:06,691 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:06,691 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:06,691 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,691 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:06,691 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:06,691 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}]]] 01:50:06,691 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 01:50:06,691 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 submitted to persistence 01:50:06,691 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 submitted to persistence 01:50:06,691 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,691 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:50:06,691 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 01:50:06,694 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}]]] 01:50:06,694 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:50:06,694 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:06,694 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=184} 01:50:06,694 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 01:50:06,694 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 01:50:06,694 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 1 01:50:06,694 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:50:06,694 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:06,694 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:06,694 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=180} 01:50:06,694 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:50:06,694 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 1 01:50:06,694 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:06,694 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:50:06,694 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 01:50:06,694 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done true 01:50:06,694 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done true 01:50:06,695 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 0, nextIndex: 1 01:50:06,695 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 0, nextIndex: 1 01:50:06,695 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:06,695 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:06,695 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 0 01:50:06,695 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 01:50:06,695 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 0 01:50:06,695 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 01:50:06,695 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data Simple{size=184} 01:50:06,695 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data Simple{size=180} 01:50:06,695 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:50:06,695 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:50:06,695 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@59d372f2}, purged=MutableUnsignedLongSet{size=0}} 01:50:06,695 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@2daf72bc}, purged=MutableUnsignedLongSet{size=0}} 01:50:06,695 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:50:06,695 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:06,695 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:50:06,695 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:50:06,695 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2: commit - successfulFromPrevious: [] 01:50:06,695 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1: commit - successfulFromPrevious: [] 01:50:06,695 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 01:50:06,695 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:50:06,695 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-726454737] with name member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 01:50:06,695 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1397568639] with name member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 01:50:06,695 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:06,695 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:06,695 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 01:50:06,695 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 01:50:06,695 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-726454737] 01:50:06,695 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:06,695 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1397568639] 01:50:06,695 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:06,695 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 65.53 μs 01:50:06,695 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 46.09 μs 01:50:06,695 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-726454737] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:06,695 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1397568639] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:06,696 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 0 01:50:06,696 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:06,696 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 0 01:50:06,696 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 01:50:06,696 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data Simple{size=184} 01:50:06,696 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 0 01:50:06,696 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:50:06,696 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:06,696 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 0 01:50:06,696 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 01:50:06,696 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data Simple{size=180} 01:50:06,696 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2}} 01:50:06,696 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=753025442136, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2}} 01:50:06,696 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:50:06,696 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} doCommit completed 01:50:06,696 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@19d000a9 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 01:50:06,696 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR}: initiating purge 01:50:06,696 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 3 01:50:06,696 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@6bb1f0e9 01:50:06,696 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@df0ad32}, purged=MutableUnsignedLongSet{size=0}} 01:50:06,696 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 01:50:06,696 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1f2b27fe}, purged=MutableUnsignedLongSet{size=0}} 01:50:06,696 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:50:06,696 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:06,696 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:50:06,696 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED}: initiating purge 01:50:06,696 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:50:06,696 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=DONE} allocated sequence 3 01:50:06,696 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@1ca50051 01:50:06,696 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1609728172] with name member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 01:50:06,696 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1722301892] with name member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 01:50:06,696 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:06,696 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1722301892] 01:50:06,696 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2de36ba9 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=753031435464} 01:50:06,696 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:06,697 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2}} 01:50:06,697 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=753025324254, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2}} 01:50:06,697 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 43.06 μs 01:50:06,697 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 01:50:06,697 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-565591775] 01:50:06,697 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1722301892] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:06,697 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2b2bc43b completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 01:50:06,697 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1609728172] 01:50:06,697 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 01:50:06,697 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 01:50:06,697 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:06,697 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@7a8f3f6e 01:50:06,697 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 01:50:06,697 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1609728172] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:06,697 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 53.92 μs 01:50:06,697 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED}: initiating purge 01:50:06,696 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 01:50:06,697 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=DONE} allocated sequence 3 01:50:06,696 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 01:50:06,697 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@1e5dbe6e 01:50:06,697 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:06,697 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:06,697 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114] 01:50:06,697 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3d381b96 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=753031946183} 01:50:06,697 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114] 01:50:06,699 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:50:06,699 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:50:06,699 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 01:50:06,699 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 01:50:06,699 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 01:50:06,699 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 01:50:06,699 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:06,699 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:06,699 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 01:50:06,699 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 01:50:06,747 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114] 01:50:06,798 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114] 01:50:06,811 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,811 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}]]] 01:50:06,812 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}]]] 01:50:06,812 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:50:06,812 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:06,812 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72} 01:50:06,812 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 01:50:06,812 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 01:50:06,812 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:06,813 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 01:50:06,813 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 01:50:06,813 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:06,813 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 1 01:50:06,813 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 01:50:06,813 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72} 01:50:06,813 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@421e2984 01:50:06,813 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:50:06,813 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:06,813 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:50:06,813 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 01:50:06,813 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:06,813 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 01:50:06,814 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 1 01:50:06,814 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:06,814 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3}} 01:50:06,814 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 1 01:50:06,814 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 01:50:06,814 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72} 01:50:06,814 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=753031946183, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3}} 01:50:06,814 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=DONE}: purge completed 01:50:06,814 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:06,814 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@497c91ad purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=DONE} 01:50:06,814 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:50:06,814 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2b2bc43b purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 01:50:06,814 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:50:06,814 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 01:50:06,814 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 01:50:06,841 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:06,841 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}]]] 01:50:06,842 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}]]] 01:50:06,842 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:50:06,842 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:06,842 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72} 01:50:06,842 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 01:50:06,842 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 01:50:06,842 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:06,844 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 01:50:06,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 01:50:06,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:06,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 1 01:50:06,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 01:50:06,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72} 01:50:06,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@635f9daa 01:50:06,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:50:06,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:50:06,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:50:06,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 01:50:06,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:06,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 01:50:06,848 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114] 01:50:06,849 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-565591775] 01:50:06,849 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3}} 01:50:06,849 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=753031435464, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3}} 01:50:06,849 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 01:50:06,849 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=DONE}: purge completed 01:50:06,849 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@21e17117 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=DONE} 01:50:06,849 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 1 01:50:06,849 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR}: purge completed 01:50:06,849 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:06,849 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@19d000a9 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 01:50:06,849 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 1 01:50:06,849 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 01:50:06,849 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72} 01:50:06,849 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392] 01:50:06,849 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:50:06,849 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392] 01:50:06,849 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:50:06,849 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 01:50:06,849 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 01:50:06,849 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@347bbdd6: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:06,849 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:06,849 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:50:06,849 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:06,849 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=OPEN} allocated sequence 0 01:50:06,850 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@347bbdd6 readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:06,850 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@461de7d4 01:50:06,850 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:06,850 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3d381b96 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=753184875600} 01:50:06,850 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE}} 01:50:06,850 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: Message size: 407 bytes, total slices to send: 1 01:50:06,850 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: Message does not need to be sliced - sending original message 01:50:06,850 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@4ba826ef 01:50:06,850 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:06,850 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2de36ba9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=753185357557} 01:50:06,850 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, message: Envelope{sessionId=1, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE}} 01:50:06,850 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: Message size: 403 bytes, total slices to send: 1 01:50:06,850 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: Message does not need to be sliced - sending original message 01:50:06,851 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:50:06,851 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:50:06,851 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 ready 01:50:06,851 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transitioned member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 to ready 01:50:06,851 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 initiating canCommit 01:50:06,851 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:50:06,851 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 validated 01:50:06,851 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:06,851 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 completed canCommit 01:50:06,852 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:50:06,852 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:50:06,852 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 ready 01:50:06,852 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transitioned member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 to ready 01:50:06,852 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 initiating canCommit 01:50:06,852 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:50:06,852 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 validated 01:50:06,852 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:06,852 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 completed canCommit 01:50:06,853 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=4, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0}} 01:50:06,853 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=753185357557, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0}} 01:50:06,853 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=FLUSHED} canCommit completed 01:50:06,853 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=4, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0}} 01:50:06,853 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=753184875600, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0}} 01:50:06,853 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=FLUSHED} canCommit completed 01:50:06,853 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@347bbdd6: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:06,853 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:06,853 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=OPEN} allocated sequence 0 01:50:06,853 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:06,853 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=OPEN} allocated sequence 0 01:50:06,854 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@347bbdd6 readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:06,854 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1810542820] 01:50:06,854 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:06,855 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428] 01:50:06,855 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428] 01:50:06,855 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-565591775] 01:50:06,855 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:06,855 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114] 01:50:06,855 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114] 01:50:06,856 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@347bbdd6: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:50:06,856 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:50:06,856 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} allocated sequence 0 01:50:06,856 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=2} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@608bcc2 01:50:06,856 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:06,856 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3d381b96 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=2}, enqueuedTicks=753191436547} 01:50:06,856 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, message: Envelope{sessionId=0, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=2}} 01:50:06,857 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: Message size: 491 bytes, total slices to send: 1 01:50:06,857 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: Message does not need to be sliced - sending original message 01:50:06,857 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} allocated sequence 1 01:50:06,857 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=2} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@be2a71c 01:50:06,857 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3d381b96 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=2}, enqueuedTicks=753192062778} 01:50:06,857 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, message: Envelope{sessionId=0, txSequence=6, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=2}} 01:50:06,857 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: Message size: 492 bytes, total slices to send: 1 01:50:06,857 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: Message does not need to be sliced - sending original message 01:50:06,857 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} allocated sequence 2 01:50:06,857 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@347bbdd6: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 01:50:06,858 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 01:50:06,858 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=OPEN} allocated sequence 0 01:50:06,858 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@347bbdd6: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:50:06,858 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:50:06,858 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=OPEN} allocated sequence 0 01:50:06,858 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1810542820] 01:50:06,858 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:50:06,859 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:06,859 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428] 01:50:06,859 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428] 01:50:06,859 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-565591775] 01:50:06,859 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:06,860 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114] 01:50:06,860 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114] 01:50:06,860 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionForwardedToLeaderAfterRetry 01:50:06,860 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=2} with Envelope{sessionId=0, txSequence=5, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0}} 01:50:06,860 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=2}, enqueuedTicks=753191436547, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0}} 01:50:06,860 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=2} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0} 01:50:06,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:06,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config null, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:06,860 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config null, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:06,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-646: Closing 01:50:06,860 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-647: Closing 01:50:06,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:50:06,860 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:50:06,861 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:06,861 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:06,861 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: removing frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 01:50:06,861 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: removing frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 01:50:06,861 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:06,861 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:06,861 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Leader to Follower 01:50:06,861 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Leader to Follower 01:50:06,861 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 01:50:06,861 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:06,861 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:06,861 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Leader to Follower 01:50:06,861 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 01:50:06,861 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:06,861 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:06,861 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Leader to Follower 01:50:06,863 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=2} with Envelope{sessionId=0, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1}} 01:50:06,863 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=2}, enqueuedTicks=753192062778, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1}} 01:50:06,863 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=2} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1} 01:50:06,910 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1810542820] 01:50:06,911 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:06,919 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428] 01:50:06,919 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@7c01af8e 01:50:06,919 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:06,919 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3d381b96 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=753254343903} 01:50:06,919 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, message: Envelope{sessionId=0, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE}} 01:50:06,920 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: Message size: 453 bytes, total slices to send: 1 01:50:06,920 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: Message does not need to be sliced - sending original message 01:50:06,920 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@23123696 01:50:06,920 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:06,920 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2de36ba9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=753254952443} 01:50:06,920 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, message: Envelope{sessionId=1, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE}} 01:50:06,920 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: Message size: 436 bytes, total slices to send: 1 01:50:06,920 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-643: Message does not need to be sliced - sending original message 01:50:06,920 PM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:50:06,920 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@347bbdd6 readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:50:06,920 PM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 01:50:06,920 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@347bbdd6 readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 01:50:06,920 PM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:50:06,920 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@347bbdd6 readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:50:06,921 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionForwardedToLeaderAfterRetry 01:50:06,922 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,922 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:06,922 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 01:50:06,922 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,922 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:50:06,922 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 01:50:06,922 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : true 01:50:06,922 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Re-initializing to Follower with previous leaderId member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:06,922 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:06,922 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:50:06,922 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : true 01:50:06,922 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:06,923 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: not currently active leader, rejecting request Envelope{sessionId=0, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE}}. isLeader: false, isLeaderActive: true,isLeadershipTransferInProgress: false, paused: false 01:50:06,924 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: request Envelope{sessionId=0, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:06,924 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: not currently active leader, rejecting request Envelope{sessionId=1, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE}}. isLeader: false, isLeaderActive: true,isLeadershipTransferInProgress: false, paused: false 01:50:06,924 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: request Envelope{sessionId=1, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:06,926 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} indicated no leadership, reconnecting it org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:06,927 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3d381b96 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@1de9f6b7, splicing queue 01:50:06,927 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:50:06,927 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3d381b96 queue spliced 1 messages 01:50:06,927 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:06,927 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: refreshing backend for shard 1 01:50:06,927 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:50:06,927 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:06,927 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:06,927 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-565591775] 01:50:06,927 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} indicated no leadership, reconnecting it org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:06,927 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2de36ba9 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@9497eeb, splicing queue 01:50:06,927 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:50:06,927 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2de36ba9 queue spliced 1 messages 01:50:06,928 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:06,928 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: refreshing backend for shard 2 01:50:06,928 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:50:06,928 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:50:06,928 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:50:06,928 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-565591775] 01:50:06,928 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:06,928 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:50:06,928 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:50:06,928 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:50:06,928 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:50:06,928 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:50:06,931 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@2563117b, attempting to connect 01:50:06,931 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@2481e1d4, attempting to connect 01:50:06,932 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 01:50:06,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 01:50:06,932 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: not currently leader, rejecting request ConnectClientRequest{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry$c], minVersion=POTASSIUM, maxVersion=POTASSIUM}. isLeader: false, isLeaderActive: true,isLeadershipTransferInProgress: false. 01:50:06,932 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: not currently leader, rejecting request ConnectClientRequest{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-people-testTransactionForwardedToLeaderAfterRetry$d], minVersion=POTASSIUM, maxVersion=POTASSIUM}. isLeader: false, isLeaderActive: true,isLeadershipTransferInProgress: false. 01:50:06,933 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Connect attempt to people failed, will retry org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:06,933 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 2 shard people failed, removing state java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:06,933 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Connect attempt to cars failed, will retry org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:06,934 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:06,934 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:06,934 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timed out resolving shard 2, scheduling retry in 1 second java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:06,934 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:06,934 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:06,971 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-565591775] 01:50:06,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:06,972 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114] 01:50:06,972 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Received TimeoutNow - switching to Candidate 01:50:06,972 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry] 01:50:06,972 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,972 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 5 01:50:06,972 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,973 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:50:06,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:50:06,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:50:06,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:06,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:06,973 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 01:50:06,973 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 01:50:06,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:06,973 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 01:50:06,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:06,975 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:06,975 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 01:50:06,975 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:06,975 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 4 - updating term 01:50:06,975 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=null 01:50:06,975 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 01:50:06,975 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,975 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:50:06,976 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:50:06,976 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:50:06,976 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry] 01:50:06,976 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:06,976 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:50:06,976 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 01:50:06,977 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 01:50:06,977 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 01:50:06,977 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:50:06,977 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:06,977 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 01:50:06,977 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:06,977 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 01:50:06,977 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:06,977 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:06,978 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done false 01:50:06,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 01:50:06,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:06,980 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done true 01:50:06,983 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:50:06,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:06,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:07,22 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-565591775] 01:50:07,22 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 01:50:07,22 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392] 01:50:07,22 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=FLUSHED} allocated sequence 1 01:50:07,22 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Received TimeoutNow - switching to Candidate 01:50:07,22 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=FLUSHED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@183db6b0 01:50:07,22 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testTransactionForwardedToLeaderAfterRetry] 01:50:07,22 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:07,22 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 5 01:50:07,22 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:07,22 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=FLUSHED} allocated sequence 1 01:50:07,22 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:07,22 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=FLUSHED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@592e777c 01:50:07,22 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:50:07,22 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:50:07,30 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:50:07,30 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:07,30 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:07,30 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 01:50:07,30 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 01:50:07,30 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:07,30 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 01:50:07,31 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@21e17117 skipping transactions [3, 4, 5] 01:50:07,31 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@22198047 aborted transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 01:50:07,31 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:50:07,31 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:07,31 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:07,31 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1657302664], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@3134db80 01:50:07,31 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-3-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 01:50:07,31 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} allocated sequence 1 01:50:07,31 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1657302664], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x00000008013037f0@98592b8 01:50:07,31 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-3-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#655979141], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:07,31 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7b5a8876 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1657302664], modifications=0, protocol=ABORT}, enqueuedTicks=753366130683} 01:50:07,31 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 01:50:07,31 PM [Cleaner-1] [DEBUG] MessageSlicer - member-3-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-635: slice: identifier: member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1657302664], modifications=0, protocol=ABORT}} 01:50:07,31 PM [Cleaner-1] [DEBUG] SlicedMessageState - member-3-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-635: Message size: 283 bytes, total slices to send: 1 01:50:07,31 PM [Cleaner-1] [DEBUG] MessageSlicer - member-3-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-635: Message does not need to be sliced - sending original message 01:50:07,31 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0} 01:50:07,31 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@545f4ab9 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=OPEN} 01:50:07,31 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@1b730350 01:50:07,31 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#655979141], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:07,31 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7d2c162b transmitting entry ConnectionEntry{request=AbortLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1197434505]}, enqueuedTicks=753366470738} 01:50:07,31 PM [Cleaner-1] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 01:50:07,31 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0} 01:50:07,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:07,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 01:50:07,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:07,32 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 4 - updating term 01:50:07,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=null 01:50:07,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 01:50:07,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:07,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:50:07,33 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:50:07,33 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:50:07,33 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-1-shard-people-testTransactionForwardedToLeaderAfterRetry] 01:50:07,33 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:07,33 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:50:07,33 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 01:50:07,33 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 01:50:07,33 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 01:50:07,33 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:50:07,33 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:07,33 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 01:50:07,33 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 01:50:07,33 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 01:50:07,34 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:07,34 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:07,34 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:50:07,34 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:07,34 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:07,34 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 01:50:07,34 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done false 01:50:07,36 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 01:50:07,36 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:07,37 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done true 01:50:07,95 PM [cluster-test-shard-dispatcher-22] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 114, lastApplied : 1, commitIndex : 1 01:50:07,145 PM [cluster-test-shard-dispatcher-22] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 107, lastApplied : 1, commitIndex : 1 01:50:07,215 PM [cluster-test-shard-dispatcher-22] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 01:50:07,285 PM [cluster-test-shard-dispatcher-22] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 138, lastApplied : 1, commitIndex : 1 01:50:07,336 PM [cluster-test-shard-dispatcher-22] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 119, lastApplied : 1, commitIndex : 1 01:50:07,452 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:50:07,457 PM [cluster-test-shard-dispatcher-22] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 119, lastApplied : 1, commitIndex : 1 01:50:07,506 PM [cluster-test-shard-dispatcher-22] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 219, lastApplied : 1, commitIndex : 1 01:50:07,575 PM [cluster-test-shard-dispatcher-22] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 116, lastApplied : 1, commitIndex : 1 01:50:07,625 PM [cluster-test-shard-dispatcher-22] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 117, lastApplied : 1, commitIndex : 1 01:50:07,695 PM [cluster-test-shard-dispatcher-22] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 119, lastApplied : 1, commitIndex : 1 01:50:07,745 PM [cluster-test-shard-dispatcher-22] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 119, lastApplied : 1, commitIndex : 1 01:50:07,815 PM [cluster-test-shard-dispatcher-22] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 119, lastApplied : 1, commitIndex : 1 01:50:07,865 PM [cluster-test-shard-dispatcher-22] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 01:50:07,935 PM [cluster-test-shard-dispatcher-22] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 01:50:07,954 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 2 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:07,954 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:50:07,954 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:50:07,954 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-565591775] 01:50:07,954 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:07,954 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:50:07,954 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:07,954 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:07,954 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-565591775] 01:50:07,954 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@129f15ff] 01:50:07,954 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:07,954 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@129f15ff] 01:50:07,954 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@19f54b2c] 01:50:07,954 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@19f54b2c] 01:50:07,954 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@29883020, attempting to connect 01:50:07,955 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6d661792, attempting to connect 01:50:07,955 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:50:07,955 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:50:07,955 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present} 01:50:07,955 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 01:50:07,955 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@21e17117 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@27cb41e8 01:50:07,955 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} previous state FLUSHED 01:50:07,955 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} previous state FLUSHED 01:50:07,955 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@27cb41e8 01:50:07,955 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:50:07,955 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=OPEN} 01:50:07,955 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Forwarding successful request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=OPEN} 01:50:07,955 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} 01:50:07,955 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} allocated sequence 0 01:50:07,955 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1247/0x0000000801301ae0@15ed06d3 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} scheduling timeout in 30 seconds 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@66f64bd3 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=true}, enqueuedTicks=753254952443} 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 replayed 1 successful requests 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Replaying queued request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} allocated sequence 1 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} enqueing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@592e777c 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@66f64bd3 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=753365927229} 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@27cb41e8 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=OPEN} 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=OPEN} 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} allocated sequence 0 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@23123696 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@66f64bd3 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=true}, enqueuedTicks=753254952443} 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@347bbdd6 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@575a2c63 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5225c5d8 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@6d9b49c, splicing queue 01:50:07,956 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5225c5d8 queue spliced 0 messages 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} in 1.452 ms 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@497c91ad instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@33e9fd4d 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} previous state FLUSHED 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} previous state FLUSHED 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} previous state SEALED 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} previous state SEALED 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} previous state SEALED 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@33e9fd4d 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=OPEN} 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Forwarding successful request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=OPEN} 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} allocated sequence 0 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1247/0x0000000801301ae0@15ed06d3 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4921ee10 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=true}, enqueuedTicks=753254343903} 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 replayed 1 successful requests 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Replaying queued request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} allocated sequence 1 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} enqueing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@183db6b0 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4921ee10 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=753357392929} 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@33e9fd4d 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=OPEN} 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=OPEN} 01:50:07,957 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1, protocol=THREE_PHASE} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} allocated sequence 0 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@7c01af8e 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4921ee10 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=true}, enqueuedTicks=753254343903} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@33e9fd4d 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Forwarding successful request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=2} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=2} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Forwarding successful request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=2} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=2} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 replayed 2 successful requests 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@33e9fd4d 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=OPEN} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=OPEN} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@33e9fd4d 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=OPEN} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=OPEN} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], modifications=1} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@347bbdd6 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@45bc9ee0 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@63759c5e superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@4733748c, splicing queue 01:50:07,958 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@63759c5e queue spliced 0 messages 01:50:07,959 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 1.806 ms 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:50:07,959 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 initiating canCommit 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 validated 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:07,959 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 initiating canCommit 01:50:07,959 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 completed canCommit 01:50:07,959 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 validated 01:50:07,959 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 initiating preCommit 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:50:07,959 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 completed canCommit 01:50:07,959 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 initiating preCommit 01:50:07,959 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 candidate ready 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2: canCommit - candidate rootPath: / 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2: canCommit - messages: [] 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2: preCommit - successfulFromPrevious: [] 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 prepared 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 completed preCommit 01:50:07,959 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 candidate ready 01:50:07,959 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1: canCommit - candidate rootPath: / 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:50:07,959 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 initiating canCommit 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 validated 01:50:07,959 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1: canCommit - messages: [] 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:07,959 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 01:50:07,959 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 completed canCommit 01:50:07,959 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 prepared 01:50:07,959 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:07,959 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 completed preCommit 01:50:07,959 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=true} with Envelope{sessionId=2, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0}} 01:50:07,960 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=true}, enqueuedTicks=753254952443, sessionId=2, txSequence=0} with Envelope{sessionId=2, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0}} 01:50:07,960 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:50:07,960 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=true} with Envelope{sessionId=3, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0}} 01:50:07,960 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 initiating canCommit 01:50:07,960 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=true}, enqueuedTicks=753254343903, sessionId=3, txSequence=0} with Envelope{sessionId=3, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0}} 01:50:07,960 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:50:07,960 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=2, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1}} 01:50:07,960 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=753365927229, sessionId=2, txSequence=1} with Envelope{sessionId=2, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1}} 01:50:07,960 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 validated 01:50:07,960 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:07,960 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:50:07,960 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 completed canCommit 01:50:07,960 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=true} with Envelope{sessionId=2, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0}} 01:50:07,960 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=true}, enqueuedTicks=753254952443, sessionId=2, txSequence=2} with Envelope{sessionId=2, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0}} 01:50:07,960 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} canCommit completed 01:50:07,960 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=3, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1}} 01:50:07,960 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=753357392929, sessionId=3, txSequence=1} with Envelope{sessionId=3, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1}} 01:50:07,960 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:50:07,960 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=true} with Envelope{sessionId=3, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0}} 01:50:07,960 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=true}, enqueuedTicks=753254343903, sessionId=3, txSequence=2} with Envelope{sessionId=3, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0}} 01:50:07,960 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} canCommit completed 01:50:07,960 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 2 01:50:07,960 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@2e0f2052 01:50:07,960 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:07,960 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:07,960 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} 01:50:07,960 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} allocated sequence 2 01:50:07,960 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@2e0f2052 01:50:07,960 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:07,960 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4921ee10 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754295382265} 01:50:07,960 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} allocated sequence 2 01:50:07,960 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@5bf29b15 01:50:07,960 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:07,960 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:07,960 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} 01:50:07,960 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} allocated sequence 2 01:50:07,960 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@5bf29b15 01:50:07,960 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 01:50:07,960 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@66f64bd3 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754295627929} 01:50:07,961 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 initiating commit 01:50:07,961 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 initiating commit 01:50:07,961 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:50:07,961 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:50:07,961 PM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 78 01:50:07,961 PM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 185 01:50:07,961 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}] 01:50:07,961 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 01:50:07,961 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}] 01:50:07,961 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 01:50:07,961 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:07,961 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:07,961 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:50:07,961 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 2 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:07,961 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:50:07,961 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 2 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:07,961 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}]]] 01:50:07,961 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}]]] 01:50:07,961 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 submitted to persistence 01:50:07,961 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 submitted to persistence 01:50:07,962 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}] 01:50:07,962 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 01:50:07,963 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}]]] 01:50:07,963 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:50:07,963 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:50:07,963 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=185} 01:50:07,963 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}]]] 01:50:07,963 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}] 01:50:07,963 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:50:07,963 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:50:07,963 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=78} 01:50:07,963 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 01:50:07,963 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 01:50:07,963 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 01:50:07,963 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:07,963 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:07,963 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}] 01:50:07,963 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 01:50:07,964 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 2, nextIndex: 3 01:50:07,964 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:50:07,964 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 2 01:50:07,964 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 01:50:07,964 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=185} 01:50:07,964 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 2, nextIndex: 3 01:50:07,964 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:50:07,964 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:50:07,964 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 2 01:50:07,964 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:50:07,964 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 01:50:07,964 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:50:07,964 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=78} 01:50:07,964 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2: commit - successfulFromPrevious: [] 01:50:07,964 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 changing state from COMMIT_PENDING to COMMITTED 01:50:07,964 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:50:07,964 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:07,964 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:50:07,964 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1: commit - successfulFromPrevious: [] 01:50:07,964 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 01:50:07,964 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 01:50:07,964 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:07,964 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:07,964 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 01:50:07,964 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:07,964 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 37.80 μs 01:50:07,964 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 01:50:07,964 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=2, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2}} 01:50:07,964 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754295627929, sessionId=2, txSequence=3} with Envelope{sessionId=2, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2}} 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} doCommit completed 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@21e17117 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR}: initiating purge 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 69.29 μs 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} allocated sequence 3 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@7c99000e 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED}: initiating purge 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=DONE} allocated sequence 3 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@707aba0f 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@66f64bd3 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754299868798} 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=3, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2}} 01:50:07,965 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754295382265, sessionId=3, txSequence=3} with Envelope{sessionId=3, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2}} 01:50:07,965 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 01:50:07,965 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} doCommit completed 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@497c91ad completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} 01:50:07,965 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:07,965 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR}: initiating purge 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 3 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@3547a880 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED}: initiating purge 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=DONE} allocated sequence 3 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@16a5756 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:07,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4921ee10 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754300392626} 01:50:07,965 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:50:07,965 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 01:50:07,965 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 01:50:07,966 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:07,966 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 01:50:07,966 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 01:50:07,967 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} allocated sequence 1 01:50:07,967 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@73486b61 01:50:07,967 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:07,967 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:07,967 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} 01:50:07,967 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} allocated sequence 1 01:50:07,967 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@73486b61 01:50:07,967 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4921ee10 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754302210266} 01:50:07,967 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} allocated sequence 1 01:50:07,967 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@2412c8ea 01:50:07,967 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:07,967 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 initiating preCommit 01:50:07,967 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:07,967 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:50:07,967 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} 01:50:07,967 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} allocated sequence 1 01:50:07,967 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@2412c8ea 01:50:07,967 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@66f64bd3 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754302343799} 01:50:07,967 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 candidate ready 01:50:07,967 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 initiating preCommit 01:50:07,967 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1: canCommit - candidate rootPath: / 01:50:07,967 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:50:07,967 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1: canCommit - messages: [] 01:50:07,967 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1: preCommit - successfulFromPrevious: [] 01:50:07,967 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 prepared 01:50:07,967 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:07,967 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 candidate ready 01:50:07,967 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 completed preCommit 01:50:07,967 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2: canCommit - candidate rootPath: / 01:50:07,967 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2: canCommit - messages: [] 01:50:07,967 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2: preCommit - successfulFromPrevious: [] 01:50:07,967 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 prepared 01:50:07,967 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:07,967 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 completed preCommit 01:50:07,968 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=3, txSequence=5, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1}} 01:50:07,968 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754302210266, sessionId=3, txSequence=5} with Envelope{sessionId=3, txSequence=5, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1}} 01:50:07,968 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:50:07,968 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=2, txSequence=5, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1}} 01:50:07,968 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754302343799, sessionId=2, txSequence=5} with Envelope{sessionId=2, txSequence=5, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1}} 01:50:07,968 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:50:07,968 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} allocated sequence 2 01:50:07,968 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@76338322 01:50:07,968 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:07,968 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:07,968 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} 01:50:07,968 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} allocated sequence 2 01:50:07,968 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@76338322 01:50:07,968 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4921ee10 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754303143742} 01:50:07,968 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} allocated sequence 2 01:50:07,968 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@4690fde6 01:50:07,968 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:07,968 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 initiating commit 01:50:07,968 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:07,968 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:50:07,968 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} 01:50:07,968 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} allocated sequence 2 01:50:07,968 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@4690fde6 01:50:07,968 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@66f64bd3 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754303272464} 01:50:07,968 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 initiating commit 01:50:07,968 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:50:07,968 PM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 237 01:50:07,968 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 01:50:07,968 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 01:50:07,968 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:07,968 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 01:50:07,968 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:07,968 PM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 223 01:50:07,968 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}], SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}]]] 01:50:07,968 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}] 01:50:07,968 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}] 01:50:07,968 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 submitted to persistence 01:50:07,968 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:07,968 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 01:50:07,968 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:07,968 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}], SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}]]] 01:50:07,968 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 submitted to persistence 01:50:07,969 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 01:50:07,969 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}] 01:50:07,971 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 2 01:50:07,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 2 01:50:07,971 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:50:07,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:50:07,971 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 2 01:50:07,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 2 01:50:07,971 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 01:50:07,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 01:50:07,971 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=185} 01:50:07,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=78} 01:50:07,971 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:50:07,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:50:07,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:07,971 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:50:07,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:50:07,971 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:50:07,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 01:50:07,971 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 01:50:07,971 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}], SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}]]] 01:50:07,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}], SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}]]] 01:50:07,971 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 01:50:07,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 01:50:07,971 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:50:07,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:50:07,971 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73} 01:50:07,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73} 01:50:07,971 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 01:50:07,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 01:50:07,971 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=223} 01:50:07,971 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}] 01:50:07,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=237} 01:50:07,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 01:50:07,971 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 01:50:07,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 01:50:07,971 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=4, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:07,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=4, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:07,971 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:07,971 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:07,972 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 21.15 μs 01:50:07,972 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 81.70 μs 01:50:07,972 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 01:50:07,972 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:07,972 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}] 01:50:07,972 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 01:50:07,972 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 4, nextIndex: 5 01:50:07,972 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 2 01:50:07,972 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 01:50:07,972 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 3 01:50:07,972 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 01:50:07,972 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73} 01:50:07,972 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 01:50:07,972 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 01:50:07,972 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@40b9eb5a 01:50:07,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:07,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 4, nextIndex: 5 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 2 01:50:07,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=2, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3}} 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 3 01:50:07,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 4 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 01:50:07,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754299868798, sessionId=2, txSequence=4} with Envelope{sessionId=2, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3}} 01:50:07,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=223} 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73} 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=DONE}: purge completed 01:50:07,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@77b8ed29 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@27cb41e8 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=DONE} 01:50:07,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR}: purge completed 01:50:07,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@21e17117 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} 01:50:07,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2: commit - successfulFromPrevious: [] 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:50:07,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 changing state from COMMIT_PENDING to COMMITTED 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 4 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=237} 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:50:07,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1: commit - successfulFromPrevious: [] 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 changing state from COMMIT_PENDING to COMMITTED 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 84.92 μs 01:50:07,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=3, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3}} 01:50:07,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754300392626, sessionId=3, txSequence=4} with Envelope{sessionId=3, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3}} 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=DONE}: purge completed 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@33e9fd4d purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=DONE} 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR}: purge completed 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@497c91ad purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=2, txSequence=6, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2}} 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754303272464, sessionId=2, txSequence=6} with Envelope{sessionId=2, txSequence=6, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2}} 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 44.96 μs 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} doCommit completed 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@21e17117 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR}: initiating purge 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} allocated sequence 3 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@24e53fb6 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-2035491948], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:07,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED}: initiating purge 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=DONE} allocated sequence 3 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@24595348 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@66f64bd3 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754308450559} 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=3, txSequence=6, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2}} 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754303143742, sessionId=3, txSequence=6} with Envelope{sessionId=3, txSequence=6, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2}} 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} doCommit completed 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@497c91ad completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR}: initiating purge 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} allocated sequence 3 01:50:07,973 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@7f51dec0 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:50:07,974 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:07,974 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 01:50:07,974 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 01:50:07,974 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED}: initiating purge 01:50:07,974 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=DONE} allocated sequence 3 01:50:07,974 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@72445613 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:07,974 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:07,974 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4921ee10 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754308741384} 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:07,974 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=FLUSHED} allocated sequence 0 01:50:07,974 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@21ff5e23 01:50:07,974 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4921ee10 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=false}, enqueuedTicks=754309423916} 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 initiating direct canCommit 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 validated 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 initiating direct preCommit 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 candidate ready 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1: canCommit - candidate rootPath: / 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1: canCommit - messages: [] 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1: preCommit - successfulFromPrevious: [] 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 prepared 01:50:07,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 initiating direct commit 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 325 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 5, leaderLastIndex: 6, leaderSnapShotIndex: 3 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 5 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=4, prevLogTerm=5, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}], SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}]]] 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 submitted to persistence 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 4 01:50:07,975 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 4 01:50:07,975 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 01:50:07,975 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 3 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 01:50:07,975 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 3 01:50:07,975 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73} 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73} 01:50:07,975 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:50:07,975 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:50:07,975 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 4 01:50:07,975 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 01:50:07,975 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=237} 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 4 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 01:50:07,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=223} 01:50:07,975 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 01:50:07,976 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:50:07,976 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:50:07,976 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:07,976 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:50:07,976 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 01:50:07,976 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 01:50:07,976 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:50:07,976 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:50:07,976 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 01:50:07,976 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 01:50:07,976 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:07,976 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 57.87 μs 01:50:07,976 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:07,976 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 27.16 μs 01:50:07,977 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 5 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:07,977 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 5 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:07,977 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=4, prevLogTerm=5, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}]]] 01:50:07,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=4, prevLogTerm=5, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}], SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}]]] 01:50:07,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 01:50:07,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 01:50:07,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73} 01:50:07,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 01:50:07,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=325} 01:50:07,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 01:50:07,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 01:50:07,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=6, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:07,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=4, prevLogTerm=5, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}]]] 01:50:07,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:50:07,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 01:50:07,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73} 01:50:07,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 01:50:07,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 01:50:07,978 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=5, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 6, nextIndex: 7 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 4 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 5 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73} 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@32ab63ed 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:50:07,978 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=3, txSequence=7, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3}} 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 6 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 5 to 6 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 6 data Simple{size=325} 01:50:07,978 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754308741384, sessionId=3, txSequence=7} with Envelope{sessionId=3, txSequence=7, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3}} 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:50:07,978 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=DONE}: purge completed 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:07,978 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@33e9fd4d purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=DONE} 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:50:07,978 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR}: purge completed 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1: commit - successfulFromPrevious: [] 01:50:07,978 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@497c91ad purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 changing state from COMMIT_PENDING to COMMITTED 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 5 for term 5 01:50:07,978 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 5, nextIndex: 6 01:50:07,978 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 01:50:07,978 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 5 01:50:07,978 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=false} with Envelope{sessionId=3, txSequence=8, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0}} 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 01:50:07,978 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 01:50:07,978 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=false}, enqueuedTicks=754309423916, sessionId=3, txSequence=8} with Envelope{sessionId=3, txSequence=8, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0}} 01:50:07,978 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73} 01:50:07,978 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@1321ecc5 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=6 01:50:07,978 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=FLUSHED} directCommit completed 01:50:07,978 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=FLUSHED}: initiating purge 01:50:07,978 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:50:07,978 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=DONE} allocated sequence 1 01:50:07,978 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@5b188b8b 01:50:07,978 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:50:07,978 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:07,978 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:50:07,978 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4921ee10 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754313490842} 01:50:07,978 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 4 for term 5 01:50:07,978 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 01:50:07,978 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:50:07,979 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=5 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:07,979 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=FLUSHED} allocated sequence 0 01:50:07,978 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=2, txSequence=7, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3}} 01:50:07,979 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@47f9f4ff 01:50:07,979 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:07,979 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754308450559, sessionId=2, txSequence=7} with Envelope{sessionId=2, txSequence=7, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3}} 01:50:07,979 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=DONE}: purge completed 01:50:07,979 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4921ee10 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=false}, enqueuedTicks=754313932279} 01:50:07,979 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@27cb41e8 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=DONE} 01:50:07,979 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR}: purge completed 01:50:07,979 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@21e17117 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} 01:50:07,979 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 55.12 μs 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 initiating direct canCommit 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 validated 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 initiating direct preCommit 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 candidate ready 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1: canCommit - candidate rootPath: / 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1: canCommit - messages: [] 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1: preCommit - successfulFromPrevious: [] 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 prepared 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 initiating direct commit 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 237 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, logIndex: 8, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 7, leaderLastIndex: 8, leaderSnapShotIndex: 5 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 7 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=6, prevLogTerm=5, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}], SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}]]] 01:50:07,979 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 submitted to persistence 01:50:07,980 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 01:50:07,980 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 01:50:07,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 01:50:07,981 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 01:50:07,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 01:50:07,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 6 01:50:07,981 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 5 01:50:07,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 4, lastIndex(): 6 01:50:07,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 5 01:50:07,981 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5 01:50:07,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 01:50:07,981 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 5 01:50:07,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73} 01:50:07,981 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 01:50:07,981 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73} 01:50:07,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:07,981 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:50:07,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:50:07,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 6 01:50:07,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 5 to 6 01:50:07,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 6 data Simple{size=325} 01:50:07,981 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:50:07,981 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 4 for term 5 01:50:07,981 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=5 01:50:07,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:50:07,982 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:07,982 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:50:07,982 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:07,982 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 5 for term 5 01:50:07,982 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 63.43 μs 01:50:07,982 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=6, prevLogTerm=5, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}], SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}]]] 01:50:07,982 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 01:50:07,982 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 01:50:07,982 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73} 01:50:07,982 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 01:50:07,982 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=237} 01:50:07,982 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 01:50:07,982 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 01:50:07,982 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=8, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:07,982 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 01:50:07,982 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 01:50:07,982 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=6 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 7 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 8, nextIndex: 9 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 8, lastAppliedIndex: 6 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 7 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 6 to 7 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 7 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73} 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@3fb1eb9b 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 8 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 7 to 8 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 8 data Simple{size=237} 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1: commit - successfulFromPrevious: [] 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 changing state from COMMIT_PENDING to COMMITTED 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 7 for term 5 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: 7 01:50:07,983 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=8 01:50:07,984 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=3, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1}} 01:50:07,984 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754313490842, sessionId=3, txSequence=9} with Envelope{sessionId=3, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1}} 01:50:07,984 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=DONE}: purge completed 01:50:07,984 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@33e9fd4d purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=DONE} 01:50:07,984 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=false} with Envelope{sessionId=3, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0}} 01:50:07,984 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=false}, enqueuedTicks=754313932279, sessionId=3, txSequence=10} with Envelope{sessionId=3, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0}} 01:50:07,984 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:07,984 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=FLUSHED} directCommit completed 01:50:07,984 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=FLUSHED}: initiating purge 01:50:07,984 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=DONE} allocated sequence 1 01:50:07,984 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=FLUSHED} allocated sequence 0 01:50:07,984 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@2531b023 01:50:07,984 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@28fc0d60 01:50:07,984 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 48.39 μs 01:50:07,984 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:07,984 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4921ee10 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754318967712} 01:50:07,984 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4921ee10 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=false}, enqueuedTicks=754319043663} 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, logIndex: 9, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 initiating direct canCommit 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 validated 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 initiating direct preCommit 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 candidate ready 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1: canCommit - candidate rootPath: / 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1: canCommit - messages: [] 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1: preCommit - successfulFromPrevious: [] 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 prepared 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 initiating direct commit 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 237 01:50:07,984 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 01:50:07,985 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 01:50:07,985 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, logIndex: 10, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:07,985 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 9, leaderLastIndex: 10, leaderSnapShotIndex: 7 01:50:07,985 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 9 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:07,985 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=8, prevLogTerm=5, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}], SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}]]] 01:50:07,985 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 submitted to persistence 01:50:07,985 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 8 01:50:07,985 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 8, context.getLastApplied(): 6, lastIndex(): 8 01:50:07,985 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 7 01:50:07,985 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 6 to 7 01:50:07,985 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 7 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73} 01:50:07,985 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:07,985 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 01:50:07,985 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:50:07,985 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 8 01:50:07,985 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 7 to 8 01:50:07,985 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 8 data Simple{size=237} 01:50:07,985 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:50:07,986 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:07,986 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:50:07,986 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 7 for term 5 01:50:07,986 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=8, prevLogTerm=5, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}], SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}]]] 01:50:07,986 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 01:50:07,986 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 8, entries to be added from: 0 01:50:07,986 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73} 01:50:07,986 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 01:50:07,986 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=237} 01:50:07,986 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 01:50:07,986 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 01:50:07,986 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=10, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:07,986 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:07,986 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 61.15 μs 01:50:07,986 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 01:50:07,986 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 01:50:07,986 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 01:50:07,986 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=8 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 9 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 10, nextIndex: 11 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 10, lastAppliedIndex: 8 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 9 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 8 to 9 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 9 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73} 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@315c80c5 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 10 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 9 to 10 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 10 data Simple{size=237} 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1: commit - successfulFromPrevious: [] 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 changing state from COMMIT_PENDING to COMMITTED 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 9 for term 5 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 11, leaderLastIndex: 10, leaderSnapShotIndex: 9 01:50:07,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=10 01:50:07,987 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=3, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1}} 01:50:07,988 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754318967712, sessionId=3, txSequence=11} with Envelope{sessionId=3, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1}} 01:50:07,988 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=DONE}: purge completed 01:50:07,988 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@33e9fd4d purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=DONE} 01:50:07,988 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=false} with Envelope{sessionId=3, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0}} 01:50:07,988 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1001741759], coordinated=false}, enqueuedTicks=754319043663, sessionId=3, txSequence=12} with Envelope{sessionId=3, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0}} 01:50:07,988 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=FLUSHED} directCommit completed 01:50:07,988 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=FLUSHED}: initiating purge 01:50:07,988 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=DONE} allocated sequence 1 01:50:07,988 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@6ae95fb7 01:50:07,988 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:07,988 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4921ee10 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754322876026} 01:50:07,988 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:07,988 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 90.77 μs 01:50:07,988 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:50:07,988 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 01:50:07,988 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 01:50:07,988 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, logIndex: 11, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:07,988 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1810542820] 01:50:07,988 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:07,988 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 01:50:07,988 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428] 01:50:07,988 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#1930306428] 01:50:07,988 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 10 01:50:07,988 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 10, context.getLastApplied(): 8, lastIndex(): 10 01:50:07,988 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 9 01:50:07,988 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 8 to 9 01:50:07,988 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 9 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73} 01:50:07,988 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:07,988 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-565591775] 01:50:07,988 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:50:07,988 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:07,988 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 10 01:50:07,988 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 9 to 10 01:50:07,988 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 10 data Simple{size=237} 01:50:07,988 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114] 01:50:07,988 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114] 01:50:07,989 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:50:07,989 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@69b72af3: creating a new snapshot member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:07,989 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:07,989 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:50:07,989 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 01:50:07,989 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:07,989 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:07,989 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1810542820] 01:50:07,989 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:07,989 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 9 for term 5 01:50:07,989 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} shard 1 01:50:07,989 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=10 01:50:07,989 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:50:07,989 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:07,989 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:07,989 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1168529124], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@6c94560d 01:50:07,989 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:50:07,989 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:50:07,989 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:07,989 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 46.68 μs 01:50:07,989 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@4f89303a, attempting to connect 01:50:07,990 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 11 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:07,990 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=10, prevLogTerm=5, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}]]] 01:50:07,991 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 01:50:07,991 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=5600, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:50:07,991 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=10, prevLogTerm=5, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}]]] 01:50:07,991 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:50:07,991 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 10, entries to be added from: 0 01:50:07,991 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73} 01:50:07,991 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 01:50:07,991 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 01:50:07,991 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=11, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:07,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 01:50:07,992 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:50:07,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 11, nextIndex: 12 01:50:07,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 11, lastAppliedIndex: 10 01:50:07,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 11 01:50:07,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 10 to 11 01:50:07,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 11 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73} 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:07,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@6951ff71 01:50:07,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@102a3572 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7c6a60a3 01:50:07,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:50:07,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:50:07,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 10 for term 5 01:50:07,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 12, leaderLastIndex: 11, leaderSnapShotIndex: 10 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7c6a60a3 01:50:07,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=11 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1168529124], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]} with Envelope{sessionId=3, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1}} 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1001741759]}, enqueuedTicks=754322876026, sessionId=3, txSequence=13} with Envelope{sessionId=3, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1}} 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=DONE}: purge completed 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@33e9fd4d purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=DONE} 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1168529124], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1248/0x0000000801301d10@5f42b927 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@89b4fcd transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1168529124], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=754324213539} 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@69b72af3 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@275b5905 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@59470e05 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@13860f8b, splicing queue 01:50:07,992 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@59470e05 queue spliced 0 messages 01:50:07,993 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 567.3 μs 01:50:07,993 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 11 01:50:07,993 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 11, context.getLastApplied(): 10, lastIndex(): 11 01:50:07,993 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 11 01:50:07,993 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 10 to 11 01:50:07,993 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 11 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73} 01:50:07,993 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:50:07,993 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:50:07,993 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 10 for term 5 01:50:07,993 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=11 01:50:07,993 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new open snapshot member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:50:07,993 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: slice: identifier: member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 01:50:07,993 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SlicedMessageState - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: Message size: 513 bytes, total slices to send: 1 01:50:07,993 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: Message does not need to be sliced - sending original message 01:50:07,995 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1168529124], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 01:50:07,995 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1168529124], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=754324213539, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 01:50:07,995 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0} 01:50:07,996 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 01:50:07,996 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:50:07,996 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:50:07,996 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1810542820] 01:50:07,996 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:50:07,996 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} shard 2 01:50:07,996 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 for transaction member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:07,996 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:50:07,996 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:50:07,996 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1168529124], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@ec75c3d 01:50:07,996 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} scheduling timeout in 30 seconds 01:50:07,996 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:50:07,996 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@647a4a01, attempting to connect 01:50:07,997 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 01:50:07,997 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=5470, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:50:07,998 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:50:07,998 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:50:07,998 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:07,998 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1d2ec644 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3c5f3553 01:50:07,998 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} previous state OPEN 01:50:07,998 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3c5f3553 01:50:07,998 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 for transaction member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:50:07,998 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} 01:50:07,998 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1168529124], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person} to successor RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} 01:50:07,998 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:50:07,998 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} enqueing request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1168529124], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1248/0x0000000801301d10@7ed08586 01:50:07,998 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:50:07,998 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3156140a transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1168529124], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person}, enqueuedTicks=754331140652} 01:50:07,998 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@69b72af3 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:07,998 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 01:50:07,998 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@74e65ade 01:50:07,998 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@60e43952 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1a627c7d, splicing queue 01:50:07,998 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@60e43952 queue spliced 0 messages 01:50:07,998 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 481.4 μs 01:50:07,999 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new open snapshot member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:50:07,999 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: slice: identifier: member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 01:50:07,999 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SlicedMessageState - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: Message size: 360 bytes, total slices to send: 1 01:50:07,999 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: Message does not need to be sliced - sending original message 01:50:08,0 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1168529124], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person} with Envelope{sessionId=1, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 01:50:08,0 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1168529124], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person}, enqueuedTicks=754331140652, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 01:50:08,0 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person completed with ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0} 01:50:08,0 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:50:08,0 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:08,0 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:08,0 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry - waiting on 2 shards 01:50:08,0 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onShutDown 01:50:08,0 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onShutDown 01:50:08,1 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:08,1 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:50:08,1 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:08,1 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:50:08,1 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 01:50:08,1 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 01:50:08,1 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All shards shutdown - sending PoisonPill to self 01:50:08,1 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 01:50:08,1 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:08,1 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:50:08,1 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: shutting down 01:50:08,1 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:08,1 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:08,2 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#-1168529124] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1168529124] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:08,2 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:08,2 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:08,6 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:08,6 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:08,6 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:08,6 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:08,7 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:08,22 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:08,23 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:08,23 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-648: Closing 01:50:08,23 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:08,23 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 01:50:08,23 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-649: Closing 01:50:08,23 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 01:50:08,23 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 01:50:08,25 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:08,25 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:09,20 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-3354945518332090119] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:09,41 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [1936055271537579471] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:09,45 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:09,46 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:09,47 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:09.048] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:09,65 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:09,66 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:09,68 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:09.070] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:09,88 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:09,93 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [3809500853443756675] 01:50:09,93 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:09,94 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:09,94 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:09,96 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:09,96 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:09,103 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:09,103 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:09,103 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:09,105 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:09,111 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [5558267619092727202] 01:50:09,111 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:09,112 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:09,112 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:09,121 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:09,121 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:09,124 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:09,136 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-5992389517385659182] 01:50:09,136 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:09,143 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:09,143 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:09,151 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:09,151 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:09,151 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:09,152 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:09,153 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,161 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-863158297] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,162 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,164 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 01:50:09,164 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:50:09,164 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@a92382f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6a8e86bd 01:50:09,164 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6eb10f34 01:50:09,164 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,164 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,165 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:09,165 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,165 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1710165009165, None)) 01:50:09,165 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165009164,0,0)) 01:50:09,165 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6a8e86bd to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@50afa082 01:50:09,165 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 01:50:09,165 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:09,166 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:09,167 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,168 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@593fc304 01:50:09,168 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:09,168 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,173 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,173 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,173 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693] 01:50:09,173 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 01:50:09,173 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1479541929] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,174 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 01:50:09,174 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,174 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#974457665 created and ready for shard:member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,175 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#974457665] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,175 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-1:snapshot-read#22843224] 01:50:09,175 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-1:snapshot-read#22843224] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,175 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:09,175 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:09,175 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 01:50:09,175 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:50:09,175 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 01:50:09,175 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 01:50:09,176 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 01:50:09,176 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,176 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 01:50:09,179 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,183 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 01:50:09,183 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:50:09,183 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@203b4290 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6b0eb353 01:50:09,183 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2e0c3056 01:50:09,184 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,184 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,184 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1710165009184, None)) 01:50:09,184 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165009183,0,0)) 01:50:09,185 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6b0eb353 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7d8db72 01:50:09,185 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 01:50:09,185 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7b3cd983 01:50:09,185 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:09,185 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,187 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 01:50:09,188 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 01:50:09,188 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-335466977] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,188 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-2:snapshot-read#-1131424941] 01:50:09,188 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-456505621 created and ready for shard:member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,188 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-456505621] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,188 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-2:snapshot-read#-1131424941] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,190 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:09,190 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:09,190 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 01:50:09,190 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 01:50:09,190 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 01:50:09,190 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:50:09,190 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 01:50:09,190 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,191 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 01:50:09,194 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:09,196 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:09,197 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:09,197 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,197 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,197 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,215 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:09,219 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:09,235 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-863158297] 01:50:09,235 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: findLocalShard : cars 01:50:09,235 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693] 01:50:09,445 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:09,445 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Received ElectionTimeout - switching to Candidate 01:50:09,446 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 01:50:09,446 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,446 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 1 01:50:09,446 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:09,446 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 01:50:09,446 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 01:50:09,446 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:50:09,446 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Follower to Candidate 01:50:09,446 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Follower to Candidate 01:50:09,724 PM [cluster-test-shard-dispatcher-16] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:09,735 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 01:50:09,735 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,735 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 2 01:50:09,995 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 01:50:09,996 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:09,996 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 3 01:50:10,117 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:10,117 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:10,117 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:10,117 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,117 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,117 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:10,117 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,117 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,118 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,118 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,127 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:10,127 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,127 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,127 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:10,127 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,127 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,127 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,162 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:10,265 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 01:50:10,265 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,265 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 4 01:50:10,265 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,265 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,267 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:10,267 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Cluster state: [] 01:50:10,267 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:10,267 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:10,267 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=null 01:50:10,267 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:10,267 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,267 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:10,267 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:10,268 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:10,268 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:10,268 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Election: Leader has following peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 01:50:10,268 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:10,268 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:10,268 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:10,268 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 01:50:10,269 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: became leader with frontend state for [] 01:50:10,269 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Candidate to Leader 01:50:10,269 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 01:50:10,269 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:10,269 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:10,269 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Candidate to Leader 01:50:10,269 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 01:50:10,269 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:50:10,269 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,269 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 01:50:10,269 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 01:50:10,269 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:10,270 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:10,270 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:10,270 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 01:50:10,270 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: -1, nextIndex: 0 01:50:10,270 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:10,293 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:10,295 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:10,297 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,300 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-430090740] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:10,301 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,305 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,305 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 01:50:10,305 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,305 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:50:10,305 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@79266d3a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@764d94c6 01:50:10,305 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@74453f1b 01:50:10,305 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1710165010305, None)) 01:50:10,305 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165010304,0,0)) 01:50:10,305 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:10,305 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,305 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:10,306 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,306 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:10,306 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,306 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@12441cfa 01:50:10,306 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@764d94c6 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@62343daf 01:50:10,306 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 01:50:10,306 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:10,306 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,306 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7c0a12e7: creating a new snapshot member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 01:50:10,306 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,306 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,307 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} 01:50:10,307 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:10,307 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1553168847] 01:50:10,307 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:10,307 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,307 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1479541929] 01:50:10,307 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,307 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1553168847] 01:50:10,307 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,307 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,307 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1553168847] 01:50:10,307 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=0} shard 1 01:50:10,307 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:10,307 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 01:50:10,307 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:10,307 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1697162047], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@a5e5378 01:50:10,307 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, primaryVersion=13] 01:50:10,307 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} scheduling timeout in 2 seconds 01:50:10,307 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, primaryVersion=13] 01:50:10,307 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 01:50:10,307 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1553168847] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:10,307 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@50ff052, attempting to connect 01:50:10,308 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-643419128 created and ready for shard:member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,308 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-643419128] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:10,308 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-3:snapshot-read#1657378155] 01:50:10,308 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-3:snapshot-read#1657378155] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:10,308 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 01:50:10,309 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} is not yet known 01:50:10,309 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, nanosAgo=18300, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:50:10,310 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:50:10,310 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:50:10,311 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:10,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2098eb2e instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7581b60e 01:50:10,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:50:10,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7581b60e 01:50:10,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 01:50:10,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 01:50:10,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1697162047], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 01:50:10,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:10,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1697162047], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1248/0x0000000801301d10@20627951 01:50:10,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 01:50:10,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13f963c5 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1697162047], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=756642260134} 01:50:10,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7c0a12e7 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:10,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} 01:50:10,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@6b308672 01:50:10,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7c555f54 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@4f4e197e, splicing queue 01:50:10,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7c555f54 queue spliced 0 messages 01:50:10,311 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:10,311 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:10,311 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 545.1 μs 01:50:10,311 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 01:50:10,311 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:50:10,311 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,311 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,311 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 01:50:10,311 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 01:50:10,317 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 01:50:10,317 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,317 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 01:50:10,318 PM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: allocating new open snapshot member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 01:50:10,318 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-651: slice: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0}} 01:50:10,318 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-651: Message size: 245 bytes, total slices to send: 1 01:50:10,318 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-651: Message does not need to be sliced - sending original message 01:50:10,319 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1697162047], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0}} 01:50:10,319 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1697162047], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=756642260134, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0}} 01:50:10,319 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0} 01:50:10,320 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:50:10,320 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,320 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader)_slicer-id-654: Closing 01:50:10,320 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 01:50:10,320 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,321 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. [WARN] [03/11/2024 13:50:10.327] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-992)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:50:10.327] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-992)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:50:10.327] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-992)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:50:10.327] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-992)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:10,342 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,343 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7c0a12e7: creating a new transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:50:10,343 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:50:10,343 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:50:10,343 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7c0a12e7 readied transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:50:10,343 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1697162047], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@180d8f6a 01:50:10,343 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:10,343 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13f963c5 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1697162047], modifications=1, protocol=SIMPLE}, enqueuedTicks=756678086324} 01:50:10,343 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-652: slice: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1697162047], modifications=1, protocol=SIMPLE}} 01:50:10,343 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 01:50:10,343 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-652: Message size: 408 bytes, total slices to send: 1 01:50:10,343 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-652: Message does not need to be sliced - sending original message 01:50:10,345 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,345 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 01:50:10,345 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,345 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 01:50:10,345 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : true 01:50:10,345 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:50:10,884 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 01:50:10,884 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 01:50:10,884 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 01:50:10,884 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Received ElectionTimeout - switching to Candidate 01:50:10,884 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 01:50:10,884 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,884 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 5 01:50:10,884 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,884 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,884 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:50:10,884 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 01:50:10,884 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 01:50:10,884 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:50:10,884 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:50:10,885 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Follower to Candidate 01:50:10,885 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=null, leaderPayloadVersion=-1] 01:50:10,885 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:10,885 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Follower to Candidate 01:50:10,885 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:10,885 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13f963c5 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@17f1bc19, splicing queue 01:50:10,885 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 01:50:10,886 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13f963c5 queue spliced 1 messages 01:50:10,886 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:10,886 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: refreshing backend for shard 1 01:50:10,886 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:50:10,886 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:10,886 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:10,886 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1479541929] 01:50:10,886 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:10,886 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Cluster state: [] 01:50:10,886 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:10,886 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:10,886 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=null 01:50:10,886 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:10,886 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:10,886 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Candidate, leaderId=null, activeMember=true] 01:50:10,886 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,886 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:50:10,888 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:50:10,888 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:50:10,888 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Election: Leader has following peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 01:50:10,888 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:10,888 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:10,888 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:50:10,888 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 01:50:10,888 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: became leader with frontend state for [] 01:50:10,888 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Candidate to Leader 01:50:10,888 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 01:50:10,888 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized - notifying 1 OnShardInitialized callbacks 01:50:10,888 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:10,888 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Candidate to Leader 01:50:10,888 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready - notifying 1 OnShardInitialized callbacks 01:50:10,888 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:10,888 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-335466977, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6b2c7d18] 01:50:10,888 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 01:50:10,888 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-335466977, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6b2c7d18] 01:50:10,889 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@67d353f3, attempting to connect 01:50:10,889 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:50:10,889 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} is not yet known 01:50:10,889 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 01:50:10,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, nanosAgo=12060, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:50:10,889 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 01:50:10,889 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:50:10,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:10,889 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:10,889 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 01:50:10,889 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-335466977], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:50:10,889 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-335466977], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:10,890 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7581b60e instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@bd5ea26 01:50:10,890 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:50:10,890 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} previous state FLUSHED 01:50:10,890 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@bd5ea26 01:50:10,890 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 01:50:10,890 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 01:50:10,892 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] LocalProxyTransaction - Not replaying IncrementTransactionSequenceRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1697162047], snapshotOnly=true} 01:50:10,893 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Incrementing sequence org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$IncrementSequence@248525d9 to successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 01:50:10,893 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 replayed 1 successful requests 01:50:10,893 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@bd5ea26 01:50:10,893 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 01:50:10,893 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=OPEN} 01:50:10,893 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1697162047], modifications=1, protocol=SIMPLE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=OPEN} 01:50:10,893 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1697162047], modifications=1, protocol=SIMPLE} 01:50:10,893 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SEALED} allocated sequence 0 01:50:10,893 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1697162047], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@180d8f6a 01:50:10,893 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-335466977], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 2 seconds 01:50:10,893 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@53e0096b transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1697162047], coordinated=false}, enqueuedTicks=756678086324} 01:50:10,893 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7c0a12e7 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-335466977], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:10,893 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: allocating new ready transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 01:50:10,893 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} 01:50:10,893 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} 01:50:10,893 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@2161477e 01:50:10,893 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2c471566 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@640f49f7, splicing queue 01:50:10,893 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2c471566 queue spliced 0 messages 01:50:10,893 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 initiating direct canCommit 01:50:10,893 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Validating transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 01:50:10,893 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 validated 01:50:10,893 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:10,893 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 initiating direct preCommit 01:50:10,893 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Preparing transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 01:50:10,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 candidate ready 01:50:10,894 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-335466977], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 3.940 ms 01:50:10,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1: canCommit - candidate rootPath: / 01:50:10,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1: canCommit - messages: [] 01:50:10,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 01:50:10,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 prepared 01:50:10,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:10,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 initiating direct commit 01:50:10,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Starting commit for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 01:50:10,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 192 01:50:10,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}] 01:50:10,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}] 01:50:10,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Replicate message: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:10,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:10,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:10,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 submitted to persistence 01:50:10,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: -1, nextIndex: 0 01:50:10,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:10,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}]]] 01:50:10,895 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}] 01:50:10,895 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done true 01:50:10,896 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}]]] 01:50:10,896 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 01:50:10,896 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:10,896 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log Simple{size=192} 01:50:10,896 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}] 01:50:10,896 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 1 01:50:10,896 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=0, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:10,896 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,896 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}] 01:50:10,896 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 0, nextIndex: 1 01:50:10,896 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:10,897 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Setting last applied to 0 01:50:10,897 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from -1 to 0 01:50:10,897 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 0 data Simple{size=192} 01:50:10,897 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Resuming commit of transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 01:50:10,897 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1f84c4cc}, purged=MutableUnsignedLongSet{size=0}} 01:50:10,897 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} 01:50:10,897 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Committed transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 01:50:10,897 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1: commit - successfulFromPrevious: [] 01:50:10,897 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 01:50:10,897 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1697162047], coordinated=false} with Envelope{sessionId=1, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0}} 01:50:10,897 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1697162047], coordinated=false}, enqueuedTicks=756678086324, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0}} 01:50:10,897 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#1180018545] with name member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher 01:50:10,897 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} directCommit completed 01:50:10,897 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR}: initiating purge 01:50:10,897 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 1 01:50:10,897 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1697162047]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@37f7bf4c 01:50:10,897 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#1180018545] 01:50:10,897 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:10,897 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:10,897 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:10,897 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1697162047]} 01:50:10,897 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:50:10,897 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SEALED}: initiating purge 01:50:10,897 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=0 01:50:10,897 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=DONE} allocated sequence 1 01:50:10,897 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1697162047]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@1c06547b 01:50:10,897 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-335466977], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:10,897 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: publishChanges: rootPath: / 01:50:10,897 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@53e0096b transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1697162047]}, enqueuedTicks=757232116072} 01:50:10,897 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: purging transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 01:50:10,897 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, size=84}] 01:50:10,897 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#1180018545] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:10,897 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, size=84}] 01:50:10,897 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Replicate message: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:10,897 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Elapsed time for generation of change events for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher: 135.1 μs 01:50:10,898 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Issuing gracefulStop to shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,898 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, size=84}] 01:50:10,898 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx - waiting on 1 shards 01:50:10,898 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onShutDown 01:50:10,898 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,898 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 01:50:10,898 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 01:50:10,899 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All shards shutdown - sending PoisonPill to self 01:50:10,899 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,899 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:50:10,899 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:50:10,899 PM [main] [WARN] ActorUtils - ShardManager for testTransactionRetryWithInitialAskTimeoutExOnCreateTx data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:50:10,900 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:50:10,900 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:50:10,899 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: shutting down 01:50:10,900 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:50:10,900 PM [main] [WARN] ActorUtils - ShardManager for testTransactionRetryWithInitialAskTimeoutExOnCreateTx data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:50:10,900 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-430090740] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-430090740] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:10,900 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:50:10,900 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-335466977] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:10,900 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-430090740] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-430090740] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:10,900 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#-546016205] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-546016205] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:10,900 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:50:10,901 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,901 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader)_slicer-id-657: Closing 01:50:10,901 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 01:50:10,901 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:10,901 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:10,901 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:11,342 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [3809500853443756675] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:11,917 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:11,917 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:50:11.921] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:11,922 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [5558267619092727202] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:11,936 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:11,936 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:11,939 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:11.940] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:11,971 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:50:11,979 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3171589246057330132] 01:50:11,979 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:11,981 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:11,981 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:11,986 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:11,986 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:11,996 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:11,996 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:11,996 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:12,2 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:12,7 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4360151525398307636] 01:50:12,7 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:12,22 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:12,22 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:12,22 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:12,22 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:12,25 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:12,35 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-4709455249492353131] 01:50:12,35 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:12,38 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:12,38 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:12,38 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:12,38 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:12,53 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:12,54 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:12,55 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithMultipleShards 01:50:12,70 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 01:50:12,75 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: no initial generation override, starting from 0 01:50:12,75 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:50:12,75 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@58fc6342 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@506a1731 01:50:12,76 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithMultipleShards 01:50:12,76 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,76 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:12,76 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:12,76 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,76 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:12,79 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@15689358 01:50:12,79 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, 0, 1710165012075, None)) 01:50:12,79 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165012074,0,0)) 01:50:12,79 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@506a1731 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@24e4d980 01:50:12,79 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} started 01:50:12,80 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:12,80 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:12,80 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@12217ecb 01:50:12,81 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:12,81 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,82 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,82 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,82 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards on shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1886548877] 01:50:12,82 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:12,82 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-1-shard-cars-testReadWriteTransactionWithMultipleShards, persistent : true 01:50:12,82 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:12,82 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:12,82 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards on shard member-1-shard-people-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#295018747] 01:50:12,82 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:12,82 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Shard created : member-1-shard-people-testReadWriteTransactionWithMultipleShards, persistent : true 01:50:12,82 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadWriteTransactionWithMultipleShards with journal batch size 1 01:50:12,83 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#-224640075 created and ready for shard:member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,83 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#2104777698] 01:50:12,84 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-854680850] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,84 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1886548877] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,84 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#295018747] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,84 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#-224640075] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,84 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#2104777698] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,84 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:12,84 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithMultipleShards 01:50:12,86 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Starting recovery for member-1-shard-people-testReadWriteTransactionWithMultipleShards with journal batch size 1 01:50:12,86 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-notifier#889136638 created and ready for shard:member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:12,86 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-notifier#889136638] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,86 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-2060255112] 01:50:12,86 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-2060255112] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,88 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:12,88 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:12,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:50:12,89 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [cars] 01:50:12,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 01:50:12,89 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is initialized 01:50:12,89 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:12,89 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:12,89 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:50:12,89 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 01:50:12,89 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [people] 01:50:12,89 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is initialized 01:50:12,89 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 01:50:12,89 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 01:50:12,89 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 01:50:12,89 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 01:50:12,89 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from null to Follower 01:50:12,89 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from null to Follower 01:50:12,94 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 01:50:12,95 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-1781981049] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,96 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:12,98 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:12,98 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: no initial generation override, starting from 0 01:50:12,98 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:50:12,98 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithMultipleShards 01:50:12,98 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@608b5065 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@59a8ff93 01:50:12,98 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,98 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@464e3fb1 01:50:12,98 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:12,99 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:12,99 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,99 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:12,99 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, 0, 1710165012099, None)) 01:50:12,99 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165012098,0,0)) 01:50:12,99 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@db28a2 01:50:12,99 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@59a8ff93 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@435583a 01:50:12,100 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} started 01:50:12,100 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:12,100 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,100 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,100 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,100 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards on shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards#907757110] 01:50:12,100 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:12,100 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-2-shard-cars-testReadWriteTransactionWithMultipleShards, persistent : true 01:50:12,100 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:12,100 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:12,100 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards on shard member-2-shard-people-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards#-1574483843] 01:50:12,101 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards#907757110] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,101 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards#-1574483843] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,101 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-people-testReadWriteTransactionWithMultipleShards, persistent : true 01:50:12,101 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#265340330 created and ready for shard:member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,101 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadWriteTransactionWithMultipleShards with journal batch size 1 01:50:12,101 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#265340330] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,101 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#1722940251] 01:50:12,101 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-520077469 created and ready for shard:member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:12,101 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-people-testReadWriteTransactionWithMultipleShards with journal batch size 1 01:50:12,101 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-520077469] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,101 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#1722940251] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,101 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:12,101 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:12,101 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#31113051] 01:50:12,102 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:50:12,102 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#31113051] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,102 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 01:50:12,102 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [cars] 01:50:12,102 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards is initialized 01:50:12,102 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,102 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 01:50:12,102 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 01:50:12,102 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:12,102 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:12,102 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [people] 01:50:12,102 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:50:12,102 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards is initialized 01:50:12,102 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 01:50:12,102 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-2-shard-cars-testReadWriteTransactionWithMultipleShards from null to Follower 01:50:12,102 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 01:50:12,102 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:12,102 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 01:50:12,102 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-2-shard-people-testReadWriteTransactionWithMultipleShards from null to Follower 01:50:12,150 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-854680850] 01:50:12,150 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: findLocalShard : cars 01:50:12,150 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1886548877] 01:50:12,342 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:12,342 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:50:12,342 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadWriteTransactionWithMultipleShards] 01:50:12,342 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,342 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 01:50:12,342 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:12,342 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Closing 01:50:12,342 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:50:12,342 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 01:50:12,342 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from Follower to Candidate 01:50:12,342 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from Follower to Candidate 01:50:12,381 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:12,381 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:50:12,381 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testReadWriteTransactionWithMultipleShards] 01:50:12,381 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:12,381 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 01:50:12,381 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:12,381 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Closing 01:50:12,381 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:50:12,381 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 01:50:12,381 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from Follower to Candidate 01:50:12,381 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from Follower to Candidate 01:50:12,621 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:50:12,621 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:12,621 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 01:50:12,661 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:50:12,661 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,661 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 01:50:12,689 PM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:12,689 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:12,891 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:50:12,891 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,891 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 01:50:12,901 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:50:12,901 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:12,901 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 01:50:12,992 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:12,992 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:12,993 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:12,993 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,993 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,993 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards 01:50:12,993 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:12,993 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:12,993 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,993 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,993 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:12,993 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:12,993 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:12,993 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Peer address for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:13,0 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:13,0 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:13,0 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:13,0 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards 01:50:13,0 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:13,0 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:13,0 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:13,0 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:13,0 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:13,0 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:13,50 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:13,201 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:50:13,201 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:13,201 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 01:50:13,201 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:50:13,201 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:13,201 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 01:50:13,201 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:13,201 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:13,203 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:13,203 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:13,203 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Cluster state: [] 01:50:13,203 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:13,203 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:13,203 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Cluster state: [] 01:50:13,203 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 01:50:13,203 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:13,203 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:13,203 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 01:50:13,203 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:13,203 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:13,203 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:13,203 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:13,203 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:13,203 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:13,204 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:13,204 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:13,204 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadWriteTransactionWithMultipleShards] 01:50:13,204 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:13,204 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:13,204 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:13,204 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:13,204 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testReadWriteTransactionWithMultipleShards] 01:50:13,204 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:50:13,204 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:13,204 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: became leader with frontend state for [] 01:50:13,204 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:50:13,204 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from Candidate to Leader 01:50:13,204 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:13,204 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:50:13,204 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:13,204 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:13,204 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from Candidate to Leader 01:50:13,204 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: became leader with frontend state for [] 01:50:13,204 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from Candidate to Leader 01:50:13,204 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:50:13,204 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:13,204 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:13,204 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from Candidate to Leader 01:50:13,204 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All Shards are ready - data store testReadWriteTransactionWithMultipleShards is ready 01:50:13,205 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Closing 01:50:13,205 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testReadWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:13,205 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:50:13,205 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:13,205 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:13,205 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Closing 01:50:13,205 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testReadWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:13,205 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithMultipleShards status sync done false 01:50:13,205 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:50:13,205 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:13,205 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:13,205 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All Shards are ready - data store testReadWriteTransactionWithMultipleShards is ready 01:50:13,206 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testReadWriteTransactionWithMultipleShards status sync done false 01:50:13,206 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:50:13,206 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:50:13,206 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:13,206 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:13,256 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-854680850] 01:50:13,256 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: findLocalShard : people 01:50:13,256 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#295018747] 01:50:13,256 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@12d1e7b8: creating a new transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:13,256 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} 01:50:13,256 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:13,257 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:13,257 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-1781981049] 01:50:13,257 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} shard 1 01:50:13,257 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:13,257 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:13,257 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} 01:50:13,257 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:50:13,257 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:50:13,257 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-1781981049] 01:50:13,257 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} shard 2 01:50:13,257 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:13,257 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:50:13,257 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@12d1e7b8 readied transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:13,257 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@5e5ab0d9 01:50:13,257 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:50:13,257 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@54940f21 01:50:13,257 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} scheduling timeout in 30 seconds 01:50:13,258 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:13,258 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 01:50:13,258 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:50:13,258 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 01:50:13,258 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 01:50:13,258 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 01:50:13,258 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@32ddff37, attempting to connect 01:50:13,258 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@3e1d6661, attempting to connect 01:50:13,260 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} is not yet known 01:50:13,260 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} is not yet known 01:50:13,260 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, nanosAgo=12110, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:50:13,260 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, nanosAgo=12560, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:50:13,261 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:50:13,261 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#295018747], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:50:13,261 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#295018747], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:13,261 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@656cadb0 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3c7e21f6 01:50:13,261 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} previous state FLUSHED 01:50:13,261 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3c7e21f6 01:50:13,261 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:50:13,261 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} 01:50:13,261 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} 01:50:13,262 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:50:13,262 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2219/0x00000008016a6000@10eec311 01:50:13,262 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:50:13,262 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#295018747], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:50:13,262 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4636ac transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=759592653796} 01:50:13,262 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-661: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], modifications=1, protocol=THREE_PHASE}} 01:50:13,262 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-661: Message size: 400 bytes, total slices to send: 1 01:50:13,262 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-661: Message does not need to be sliced - sending original message 01:50:13,262 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@12d1e7b8 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#295018747], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:13,262 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 01:50:13,262 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@7487d1e3 01:50:13,262 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4500e23 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3ad23d02, splicing queue 01:50:13,262 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4500e23 queue spliced 0 messages 01:50:13,262 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#295018747], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 969.7 μs 01:50:13,262 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1886548877], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:50:13,263 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1886548877], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:13,263 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2460f225 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4d46b35b 01:50:13,263 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:50:13,263 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4d46b35b 01:50:13,263 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:50:13,263 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} 01:50:13,263 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} 01:50:13,263 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:13,263 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2219/0x00000008016a6000@627293c5 01:50:13,263 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1886548877], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:50:13,263 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@46c6f5e0 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=759592546564} 01:50:13,263 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-661: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, message: Envelope{sessionId=1, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], modifications=1, protocol=THREE_PHASE}} 01:50:13,263 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-661: Message size: 397 bytes, total slices to send: 1 01:50:13,263 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-661: Message does not need to be sliced - sending original message 01:50:13,263 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@12d1e7b8 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1886548877], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:13,263 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 01:50:13,263 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@64c84294 01:50:13,263 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@499db68a superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@392778b6, splicing queue 01:50:13,263 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@499db68a queue spliced 0 messages 01:50:13,263 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1886548877], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 795.5 μs 01:50:13,264 PM [cluster-test-shard-dispatcher-14] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testReadWriteTransactionWithMultipleShards: allocating new open transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:50:13,264 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: allocating new open transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:50:13,264 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:50:13,264 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 ready 01:50:13,264 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:50:13,264 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: transitioned member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 to ready 01:50:13,264 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 ready 01:50:13,265 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating canCommit 01:50:13,265 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:50:13,265 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: transitioned member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 to ready 01:50:13,265 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating canCommit 01:50:13,265 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 validated 01:50:13,265 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:13,265 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:50:13,265 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 validated 01:50:13,265 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:13,265 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed canCommit 01:50:13,265 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed canCommit 01:50:13,266 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0}} 01:50:13,266 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=759592653796, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0}} 01:50:13,266 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} canCommit completed 01:50:13,266 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0}} 01:50:13,266 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=759592546564, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0}} 01:50:13,266 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 01:50:13,266 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:50:13,266 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@66ced0c4 01:50:13,266 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} 01:50:13,266 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} allocated sequence 1 01:50:13,266 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1242/0x0000000801303c28@6d9978bc 01:50:13,266 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1886548877], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:13,266 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@46c6f5e0 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]}, enqueuedTicks=759601352170} 01:50:13,266 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 01:50:13,266 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@19808f29 01:50:13,266 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} 01:50:13,266 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} allocated sequence 1 01:50:13,266 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1242/0x0000000801303c28@64a1096e 01:50:13,266 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#295018747], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:13,266 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4636ac transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]}, enqueuedTicks=759601572264} 01:50:13,267 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating preCommit 01:50:13,267 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating preCommit 01:50:13,267 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:50:13,267 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:50:13,267 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 candidate ready 01:50:13,267 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1: canCommit - candidate rootPath: / 01:50:13,267 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 candidate ready 01:50:13,267 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2: canCommit - candidate rootPath: / 01:50:13,267 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1: canCommit - messages: [] 01:50:13,267 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 01:50:13,267 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 prepared 01:50:13,267 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:13,267 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed preCommit 01:50:13,267 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2: canCommit - messages: [] 01:50:13,267 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2: preCommit - successfulFromPrevious: [] 01:50:13,267 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 prepared 01:50:13,267 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:13,267 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed preCommit 01:50:13,268 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1}} 01:50:13,268 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]}, enqueuedTicks=759601352170, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1}} 01:50:13,268 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:50:13,268 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1}} 01:50:13,268 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]}, enqueuedTicks=759601572264, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1}} 01:50:13,268 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:50:13,268 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:50:13,268 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@52837775 01:50:13,268 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} 01:50:13,268 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} allocated sequence 2 01:50:13,268 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@52837775 01:50:13,268 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1886548877], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:13,268 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@46c6f5e0 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]}, enqueuedTicks=759603369203} 01:50:13,268 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 2 01:50:13,268 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@4f6753cb 01:50:13,268 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} 01:50:13,268 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} allocated sequence 2 01:50:13,268 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@4f6753cb 01:50:13,268 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#295018747], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:13,268 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4636ac transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]}, enqueuedTicks=759603561166} 01:50:13,269 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating commit 01:50:13,269 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:50:13,269 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating commit 01:50:13,269 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:50:13,269 PM [cluster-test-shard-dispatcher-14] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 180 01:50:13,269 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 184 01:50:13,269 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 01:50:13,269 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:50:13,269 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 01:50:13,269 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:50:13,269 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:13,269 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:13,269 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:13,269 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:13,269 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:13,269 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}]]] 01:50:13,269 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:13,269 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 submitted to persistence 01:50:13,270 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 01:50:13,270 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 submitted to persistence 01:50:13,270 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}]]] 01:50:13,270 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:50:13,271 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:13,271 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=184} 01:50:13,271 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 01:50:13,271 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 1 01:50:13,271 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:13,271 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 01:50:13,271 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:50:13,271 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:13,271 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=180} 01:50:13,271 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:50:13,271 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 1 01:50:13,271 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:13,271 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:50:13,271 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:50:13,271 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:13,271 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:13,274 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 01:50:13,274 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:13,274 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:50:13,274 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 01:50:13,274 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:50:13,274 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=184} 01:50:13,274 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:13,274 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:50:13,274 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 01:50:13,274 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:50:13,274 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=180} 01:50:13,274 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@664d9528}, purged=MutableUnsignedLongSet{size=0}} 01:50:13,274 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:50:13,274 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 01:50:13,274 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:50:13,274 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2: commit - successfulFromPrevious: [] 01:50:13,274 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 01:50:13,274 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@333189c3}, purged=MutableUnsignedLongSet{size=0}} 01:50:13,274 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 01:50:13,274 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:50:13,274 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1: commit - successfulFromPrevious: [] 01:50:13,274 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:50:13,274 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1255607121] with name member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher 01:50:13,274 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#854482751] with name member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher 01:50:13,274 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:50:13,274 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1255607121] 01:50:13,274 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:50:13,274 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:50:13,274 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#854482751] 01:50:13,274 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1255607121] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:13,274 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 53.33 μs 01:50:13,274 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#854482751] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 99.48 μs 01:50:13,275 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 01:50:13,275 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testReadWriteTransactionWithMultipleShards status sync done true 01:50:13,275 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:50:13,275 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithMultipleShards status sync done true 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2}} 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]}, enqueuedTicks=759603561166, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2}} 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} doCommit completed 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@656cadb0 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR}: initiating purge 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 3 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@2b41ac19 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED}: initiating purge 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} allocated sequence 3 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@d3b9707 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#295018747], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4636ac transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]}, enqueuedTicks=759610446209} 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2}} 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]}, enqueuedTicks=759603369203, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2}} 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2460f225 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@2bd67694 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED}: initiating purge 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} allocated sequence 3 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@dd12693 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1886548877], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:13,275 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@46c6f5e0 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]}, enqueuedTicks=759610615922} 01:50:13,275 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@12d1e7b8: creating a new snapshot member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:50:13,275 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:50:13,276 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:50:13,276 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@26329b1b 01:50:13,276 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@46c6f5e0 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=759610817975} 01:50:13,276 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: purging transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:50:13,276 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 01:50:13,276 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 01:50:13,276 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:13,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: purging transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:50:13,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 01:50:13,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 01:50:13,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:13,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: allocating new open snapshot member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1 01:50:13,282 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithMultipleShards_slicer-id-659: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, message: Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 01:50:13,282 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 01:50:13,282 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 01:50:13,282 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-1-shard-cars-testReadWriteTransactionWithMultipleShards_slicer-id-659: Message size: 337 bytes, total slices to send: 1 01:50:13,282 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithMultipleShards_slicer-id-659: Message does not need to be sliced - sending original message 01:50:13,283 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 01:50:13,283 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=759610817975, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 01:50:13,283 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0} 01:50:13,283 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:50:13,283 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} allocated sequence 0 01:50:13,283 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@4d27c8f9 01:50:13,283 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4636ac transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people}, enqueuedTicks=759618030875} 01:50:13,283 PM [cluster-test-shard-dispatcher-14] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testReadWriteTransactionWithMultipleShards: allocating new open snapshot member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2 01:50:13,283 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-people-testReadWriteTransactionWithMultipleShards_slicer-id-660: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, message: Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 01:50:13,283 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SlicedMessageState - member-1-shard-people-testReadWriteTransactionWithMultipleShards_slicer-id-660: Message size: 341 bytes, total slices to send: 1 01:50:13,283 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-people-testReadWriteTransactionWithMultipleShards_slicer-id-660: Message does not need to be sliced - sending original message 01:50:13,284 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people} with Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 01:50:13,284 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people}, enqueuedTicks=759618030875, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 01:50:13,284 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people completed with ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0} 01:50:13,284 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:50:13,284 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:13,284 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:13,284 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithMultipleShards - waiting on 2 shards 01:50:13,284 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onShutDown 01:50:13,284 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onShutDown 01:50:13,284 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Initiating leader transfer 01:50:13,284 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Initiating leader transfer 01:50:13,285 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@36e89c12 01:50:13,285 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@36e89c12 01:50:13,285 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 01:50:13,285 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@188b08d7 01:50:13,285 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:13,285 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:13,285 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}]]] 01:50:13,285 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@188b08d7 01:50:13,285 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 01:50:13,285 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:50:13,285 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:13,285 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:13,285 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:13,285 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:13,285 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}]]] 01:50:13,285 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:50:13,285 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:13,285 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:13,286 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards] 01:50:13,286 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards] 01:50:13,286 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:50:13,286 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:13,287 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}]]] 01:50:13,287 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 01:50:13,287 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:50:13,287 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:13,287 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:50:13,287 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72} 01:50:13,287 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 01:50:13,287 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:13,287 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 2 01:50:13,287 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4636ac superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@26cbbd04, splicing queue 01:50:13,287 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 0 01:50:13,287 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}]]] 01:50:13,287 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:50:13,287 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:13,287 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:13,287 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72} 01:50:13,287 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 01:50:13,287 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#295018747], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:50:13,287 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:50:13,287 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 2 01:50:13,287 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 01:50:13,287 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 0 01:50:13,287 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:50:13,287 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=184} 01:50:13,287 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4636ac queue spliced 1 messages 01:50:13,287 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:13,287 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:50:13,287 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#295018747], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#295018747], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:13,287 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 01:50:13,287 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:50:13,287 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:50:13,287 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=180} 01:50:13,287 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: refreshing backend for shard 2 01:50:13,287 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#295018747], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:50:13,287 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:50:13,287 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:50:13,287 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:50:13,287 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-1781981049] 01:50:13,287 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@def050a}, purged=MutableUnsignedLongSet{size=0}} 01:50:13,287 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@296e963}, purged=MutableUnsignedLongSet{size=0}} 01:50:13,287 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 01:50:13,287 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 01:50:13,287 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:50:13,287 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:50:13,287 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1472756662] with name member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher 01:50:13,287 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1633829309] with name member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher 01:50:13,288 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:50:13,288 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:50:13,288 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1472756662] 01:50:13,288 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:50:13,288 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 51.85 μs 01:50:13,288 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1472756662] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:13,288 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1633829309] 01:50:13,288 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:50:13,288 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 29.40 μs 01:50:13,288 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1633829309] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:13,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 01:50:13,290 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 01:50:13,290 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:50:13,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:50:13,293 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:13,293 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@46c6f5e0 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@2d837a5a, splicing queue 01:50:13,293 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1886548877], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:50:13,293 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@46c6f5e0 queue spliced 1 messages 01:50:13,293 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1886548877], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1886548877], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:13,293 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: refreshing backend for shard 1 01:50:13,294 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1886548877], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:50:13,294 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:13,294 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:13,294 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-1781981049] 01:50:13,294 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:50:13,294 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72} 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: replication of member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@2788c082 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testReadWriteTransactionWithMultipleShards: finished purging transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testReadWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Leader transfer complete 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72} 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: replication of member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@11db6f7e 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: finished purging transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:50:13,294 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:50:13,295 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:13,295 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:50:13,295 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:50:13,295 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:13,295 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Leader transfer complete 01:50:13,295 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:50:13,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 1 01:50:13,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:13,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 01:50:13,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:50:13,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72} 01:50:13,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 01:50:13,296 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} not found in inflight queue, checking pending queue 01:50:13,296 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} 01:50:13,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:50:13,296 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]}, enqueuedTicks=759610446209, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} 01:50:13,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:50:13,296 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE}: purge completed 01:50:13,296 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3c7e21f6 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} 01:50:13,296 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR}: purge completed 01:50:13,296 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@656cadb0 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 01:50:13,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:50:13,296 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} not found in inflight queue, checking pending queue 01:50:13,296 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} 01:50:13,296 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1366754220]}, enqueuedTicks=759610615922, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} 01:50:13,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 1 01:50:13,296 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE}: purge completed 01:50:13,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:13,296 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4d46b35b purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} 01:50:13,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 01:50:13,296 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:50:13,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:50:13,296 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2460f225 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 01:50:13,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72} 01:50:13,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 01:50:13,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:50:13,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:50:13,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:50:13,701 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 01:50:13,701 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1886548877]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:13,702 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:13,702 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:13,702 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader)_slicer-id-664: Closing 01:50:13,702 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader)_slicer-id-665: Closing 01:50:13,702 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:50:13,702 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:50:13,702 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 01:50:13,702 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 01:50:13,702 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:50:13,702 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: shutting down 01:50:13,702 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 01:50:13,702 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 01:50:13,703 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-854680850] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-854680850] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:13,703 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:13,703 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:13,708 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:13,708 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:13,708 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:13,708 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:13,709 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:13,722 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 01:50:13,722 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:13,722 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Closing 01:50:13,722 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:50:13,723 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:13,723 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Closing 01:50:13,723 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:50:13,723 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 01:50:13,723 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:13,723 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:14,719 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-3171589246057330132] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:14,739 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4360151525398307636] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:14,742 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:14,742 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:14,743 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:14.745] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:14,752 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:14,752 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:14,754 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:14.755] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:14,767 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:14,772 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [8493403872891738569] 01:50:14,772 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:14,774 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:14,774 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:14,774 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:14,774 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:14,784 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:14,784 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:14,784 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:14,787 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:14,790 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-948099058454757733] 01:50:14,791 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:14,792 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:14,792 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:14,797 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:14,797 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:14,804 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:14,811 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-6650596113484247041] 01:50:14,811 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:14,816 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:14,816 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:14,816 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:14,816 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:14,819 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:14,821 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:14,829 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@102a3572 aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 01:50:14,829 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:50:14,829 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1168529124], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@5bbee93b 01:50:14,829 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 01:50:14,829 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} allocated sequence 1 01:50:14,829 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1168529124], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x00000008013037f0@14ede622 01:50:14,829 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#195828114], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:14,829 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@89b4fcd transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1168529124], modifications=0, protocol=ABORT}, enqueuedTicks=761164369641} 01:50:14,829 PM [Cleaner-1] [DEBUG] MessageSlicer - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-640: slice: identifier: member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1168529124], modifications=0, protocol=ABORT}} 01:50:14,829 PM [Cleaner-1] [DEBUG] SlicedMessageState - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-640: Message size: 293 bytes, total slices to send: 1 01:50:14,829 PM [Cleaner-1] [DEBUG] MessageSlicer - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-640: Message does not need to be sliced - sending original message 01:50:14,829 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1d2ec644 aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 01:50:14,829 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 01:50:14,829 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1168529124], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@2b809f05 01:50:14,829 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 01:50:14,829 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} allocated sequence 1 01:50:14,829 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} sending request ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1168529124], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x00000008013037f0@f68c370 01:50:14,829 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1737764392], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:14,829 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3156140a transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1168529124], modifications=0, protocol=ABORT}, enqueuedTicks=761164725247} 01:50:14,829 PM [Cleaner-1] [DEBUG] MessageSlicer - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-640: slice: identifier: member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1168529124], modifications=0, protocol=ABORT}} 01:50:14,830 PM [Cleaner-1] [DEBUG] SlicedMessageState - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-640: Message size: 294 bytes, total slices to send: 1 01:50:14,830 PM [Cleaner-1] [DEBUG] MessageSlicer - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-640: Message does not need to be sliced - sending original message 01:50:14,830 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0} 01:50:14,830 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2098eb2e aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} 01:50:14,830 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:50:14,830 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1697162047], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@23e92a7c 01:50:14,830 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} 01:50:14,830 PM [Cleaner-1] [DEBUG] RemoteProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} has a successor, which should receive seal through a separate request 01:50:14,830 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:50:14,830 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1697162047], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x00000008013037f0@7f1ac404 01:50:14,830 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:14,830 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1345008693], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:14,830 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@bd5ea26 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 01:50:14,830 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1697162047]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@3bb84e37 01:50:14,830 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@53e0096b transmitting entry ConnectionEntry{request=AbortLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1697162047]}, enqueuedTicks=761165101863} 01:50:14,830 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0} 01:50:14,830 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4d46b35b aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} 01:50:14,830 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} allocated sequence 1 01:50:14,830 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@34ba3a14 01:50:14,830 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1886548877], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:14,830 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1886548877], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:14,830 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:14,830 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3c7e21f6 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} 01:50:14,830 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} allocated sequence 1 01:50:14,830 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1366754220], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@5eea4c35 01:50:14,830 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#295018747], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:14,830 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#295018747], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:50:14,830 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0} 01:50:14,831 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:14,832 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLargeReadReplySlicing 01:50:14,846 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1258405347] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:14,848 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testLargeReadReplySlicing 01:50:14,849 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: no initial generation override, starting from 0 01:50:14,849 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testLargeReadReplySlicing, generation=0} 01:50:14,849 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLargeReadReplySlicing: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2fda100e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1bb60a7 01:50:14,849 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@44b9fef4 01:50:14,850 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testLargeReadReplySlicing, 0, 1710165014850, None)) 01:50:14,850 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testLargeReadReplySlicing 01:50:14,850 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Creating local shard: member-1-shard-cars-testLargeReadReplySlicing 01:50:14,850 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:14,850 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 01:50:14,851 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165014849,0,0)) 01:50:14,851 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLargeReadReplySlicing: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1bb60a7 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6e5a08f9 01:50:14,851 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testLargeReadReplySlicing, generation=0} started 01:50:14,851 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@602402d4 01:50:14,852 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:14,852 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testLargeReadReplySlicing 01:50:14,852 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:14,852 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 01:50:14,852 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 01:50:14,852 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: updated peer member-1-shard-cars-testLargeReadReplySlicing on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing on shard member-1-shard-cars-testLargeReadReplySlicing whose actor address is Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251] 01:50:14,853 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testLargeReadReplySlicing, persistent : true 01:50:14,853 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testLargeReadReplySlicing with journal batch size 1 01:50:14,854 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:14,854 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:14,854 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing-notifier#-1126181490 created and ready for shard:member-1-shard-cars-testLargeReadReplySlicing 01:50:14,854 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing-notifier#-1126181490] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:14,854 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing/shard-member-1:snapshot-read#-1681758799] 01:50:14,854 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing/shard-member-1:snapshot-read#-1681758799] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:14,856 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLargeReadReplySlicing 01:50:14,857 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:14,857 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:14,857 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLargeReadReplySlicing: onLeadershipChange, isLeader: false, hasLeader : false 01:50:14,857 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLargeReadReplySlicing is no longer the leader 01:50:14,857 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Initializing shard [cars] 01:50:14,857 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-1-shard-cars-testLargeReadReplySlicing is initialized 01:50:14,857 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLargeReadReplySlicing , received role change from null to Follower 01:50:14,857 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLargeReadReplySlicing , registered listener akka://cluster-test/user/shardmanager-testLargeReadReplySlicing 01:50:14,857 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received role changed for member-1-shard-cars-testLargeReadReplySlicing from null to Follower 01:50:14,862 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testLargeReadReplySlicing 01:50:14,864 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-135333977] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:14,873 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:14,874 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: no initial generation override, starting from 0 01:50:14,874 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} 01:50:14,874 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLargeReadReplySlicing: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7ddb7626 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7995ef5c 01:50:14,874 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@257be47 01:50:14,874 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:14,875 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testLargeReadReplySlicing, 0, 1710165014874, None)) 01:50:14,875 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testLargeReadReplySlicing 01:50:14,875 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Creating local shard: member-2-shard-cars-testLargeReadReplySlicing 01:50:14,876 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:14,876 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 01:50:14,876 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165014873,0,0)) 01:50:14,876 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLargeReadReplySlicing: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7995ef5c to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@971826 01:50:14,876 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} started 01:50:14,877 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1a99a53b 01:50:14,877 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:14,877 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testLargeReadReplySlicing 01:50:14,877 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 01:50:14,877 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 01:50:14,877 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: updated peer member-1-shard-cars-testLargeReadReplySlicing on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing on shard member-2-shard-cars-testLargeReadReplySlicing whose actor address is Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing#-2000881514] 01:50:14,877 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testLargeReadReplySlicing, persistent : true 01:50:14,878 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testLargeReadReplySlicing with journal batch size 1 01:50:14,878 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing#-2000881514] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:14,878 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing/shard-member-2:snapshot-read#2036481916] 01:50:14,878 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing-notifier#2084277713] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:14,879 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:14,879 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testLargeReadReplySlicing: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:14,879 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLargeReadReplySlicing: onLeadershipChange, isLeader: false, hasLeader : false 01:50:14,879 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLargeReadReplySlicing is no longer the leader 01:50:14,879 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Initializing shard [cars] 01:50:14,879 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLargeReadReplySlicing is initialized 01:50:14,879 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLargeReadReplySlicing set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 01:50:14,881 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing-notifier#2084277713 created and ready for shard:member-2-shard-cars-testLargeReadReplySlicing 01:50:14,881 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLargeReadReplySlicing , received role change from null to Follower 01:50:14,881 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLargeReadReplySlicing , registered listener akka://cluster-test/user/shardmanager-testLargeReadReplySlicing 01:50:14,881 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing/shard-member-2:snapshot-read#2036481916] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:14,881 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received role changed for member-2-shard-cars-testLargeReadReplySlicing from null to Follower 01:50:14,927 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1258405347] 01:50:14,927 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: findLocalShard : cars 01:50:14,927 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251] 01:50:15,163 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:15,163 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Follower): Received ElectionTimeout - switching to Candidate 01:50:15,163 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testLargeReadReplySlicing] 01:50:15,163 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Set currentTerm=1, votedFor=member-1-shard-cars-testLargeReadReplySlicing 01:50:15,163 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Starting new election term 1 01:50:15,163 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:15,163 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing (Follower): Closing 01:50:15,163 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLargeReadReplySlicing: onLeadershipChange, isLeader: false, hasLeader : false 01:50:15,163 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLargeReadReplySlicing is no longer the leader 01:50:15,164 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLargeReadReplySlicing , received role change from Follower to Candidate 01:50:15,164 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received role changed for member-1-shard-cars-testLargeReadReplySlicing from Follower to Candidate 01:50:15,446 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:15,454 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Received ElectionTimeout 01:50:15,454 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Set currentTerm=2, votedFor=member-1-shard-cars-testLargeReadReplySlicing 01:50:15,454 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Starting new election term 2 01:50:15,713 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Received ElectionTimeout 01:50:15,713 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Set currentTerm=3, votedFor=member-1-shard-cars-testLargeReadReplySlicing 01:50:15,713 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Starting new election term 3 01:50:15,794 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:15,794 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:15,795 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:15,795 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLargeReadReplySlicing/member-3-shard-cars-testLargeReadReplySlicing 01:50:15,795 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLargeReadReplySlicing/member-3-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 01:50:15,795 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:15,795 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 01:50:15,795 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 01:50:15,795 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLargeReadReplySlicing set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 01:50:15,800 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:15,800 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLargeReadReplySlicing/member-3-shard-cars-testLargeReadReplySlicing 01:50:15,800 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLargeReadReplySlicing/member-3-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 01:50:15,800 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:15,800 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 01:50:15,800 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 01:50:15,832 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:15,953 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Received ElectionTimeout 01:50:15,953 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Set currentTerm=4, votedFor=member-1-shard-cars-testLargeReadReplySlicing 01:50:15,953 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Starting new election term 4 01:50:15,953 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLargeReadReplySlicing, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testLargeReadReplySlicing 01:50:15,954 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:15,954 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): Cluster state: [] 01:50:15,954 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testLargeReadReplySlicing, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:15,954 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testLargeReadReplySlicing, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:15,954 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: Set currentTerm=4, votedFor=null 01:50:15,954 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testLargeReadReplySlicing, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:15,954 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: Set currentTerm=4, votedFor=member-1-shard-cars-testLargeReadReplySlicing 01:50:15,954 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:15,956 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:15,956 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:15,956 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Election: Leader has following peers: [member-2-shard-cars-testLargeReadReplySlicing] 01:50:15,956 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLargeReadReplySlicing: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:15,956 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:15,956 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLargeReadReplySlicing: onLeadershipChange, isLeader: true, hasLeader : true 01:50:15,956 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: became leader with frontend state for [] 01:50:15,956 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLargeReadReplySlicing , received role change from Candidate to Leader 01:50:15,956 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLargeReadReplySlicing, leaderId=member-1-shard-cars-testLargeReadReplySlicing, leaderPayloadVersion=13] 01:50:15,956 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:15,956 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:15,956 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received role changed for member-1-shard-cars-testLargeReadReplySlicing from Candidate to Leader 01:50:15,956 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: All Shards are ready - data store testLargeReadReplySlicing is ready 01:50:15,964 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testLargeReadReplySlicing: Closing 01:50:15,964 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: getLeaderAddress leaderId = member-1-shard-cars-testLargeReadReplySlicing peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 01:50:15,964 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing Received follower initial sync status for member-2-shard-cars-testLargeReadReplySlicing status sync done false 01:50:15,965 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLargeReadReplySlicing, leaderId=member-1-shard-cars-testLargeReadReplySlicing, leaderPayloadVersion=13] 01:50:15,965 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:15,965 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:15,965 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: All Shards are ready - data store testLargeReadReplySlicing is ready 01:50:15,966 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLargeReadReplySlicing updated: matchIndex: -1, nextIndex: 0 01:50:15,966 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLargeReadReplySlicing: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:15,982 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@13517e73: creating a new transaction member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-0 01:50:15,983 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} 01:50:15,983 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:15,983 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:15,983 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-135333977] 01:50:15,983 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, history=0, cookie=0} shard 1 01:50:15,983 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:15,983 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1 for transaction member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-0 01:50:15,983 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:15,983 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing, primaryVersion=13] 01:50:15,983 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} allocated sequence 1 01:50:15,983 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#725556861], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@13d56d02 01:50:15,983 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:50:15,983 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#725556861], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@21fae753 01:50:15,983 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing, primaryVersion=13] 01:50:15,984 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@13e39eed, attempting to connect 01:50:15,985 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: client ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} is not yet known 01:50:15,985 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, nanosAgo=9640, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} 01:50:15,991 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:50:15,991 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:50:15,991 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:15,991 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4150f92f instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@79612113 01:50:15,991 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:50:15,991 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@79612113 01:50:15,991 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1 for transaction member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1 01:50:15,991 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} 01:50:15,991 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#725556861], modifications=1} to successor RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} 01:50:15,991 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:15,991 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#725556861], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} 01:50:15,991 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} enqueing request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#725556861], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2250/0x00000008016bde88@13e3d635 01:50:15,991 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:50:15,991 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6f0c75c2 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#725556861], modifications=1}, enqueuedTicks=762318444613} 01:50:15,991 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-668: slice: identifier: member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#725556861], modifications=1}} 01:50:15,992 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-668: Message size: 448 bytes, total slices to send: 5 01:50:15,992 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-668: getNextSlice: total size: 448, offset: 0, size: 100, index: 1 01:50:15,999 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-668: Sending first slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76], data.length=100, sliceIndex=1, totalSlices=5, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test/user/$a#725556861]] 01:50:15,999 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} allocated sequence 1 01:50:15,999 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#725556861], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1248/0x0000000801301d10@3ec1e636 01:50:15,999 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6f0c75c2 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#725556861], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=762318444613} 01:50:15,999 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6f0c75c2 cannot transmit request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#725556861], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} - delaying it 01:50:15,999 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@13517e73 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:16,0 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} 01:50:16,0 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@3e0af53e 01:50:16,0 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1b6372b5 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@27fd8048, splicing queue 01:50:16,0 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1b6372b5 queue spliced 0 messages 01:50:16,0 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 8.661 ms 01:50:16,5 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76], data.length=100, sliceIndex=1, totalSlices=5, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#725556861]] 01:50:16,6 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Received first slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76] - creating AssembledMessageState 01:50:16,8 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76], sliceIndex: 1, lastSliceIndex: 0, assembledSize: 0, sliceHashCode: -1, lastSliceHashCode: -1 01:50:16,8 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76] - expecting more 01:50:16,13 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-668: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76], sliceIndex=1, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251]] 01:50:16,13 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-668: getNextSlice: total size: 448, offset: 100, size: 100, index: 2 01:50:16,13 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-668: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76], data.length=100, sliceIndex=2, totalSlices=5, lastSliceHashCode=-1085826726, replyTo=Actor[akka://cluster-test/user/$a#725556861]] 01:50:16,13 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76], data.length=100, sliceIndex=2, totalSlices=5, lastSliceHashCode=-1085826726, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#725556861]] 01:50:16,14 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76], sliceIndex: 2, lastSliceIndex: 1, assembledSize: 100, sliceHashCode: -1085826726, lastSliceHashCode: -1085826726 01:50:16,14 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76] - expecting more 01:50:16,14 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-668: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76], sliceIndex=2, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251]] 01:50:16,14 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-668: getNextSlice: total size: 448, offset: 200, size: 100, index: 3 01:50:16,14 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-668: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76], data.length=100, sliceIndex=3, totalSlices=5, lastSliceHashCode=-840689321, replyTo=Actor[akka://cluster-test/user/$a#725556861]] 01:50:16,15 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76], data.length=100, sliceIndex=3, totalSlices=5, lastSliceHashCode=-840689321, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#725556861]] 01:50:16,15 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76], sliceIndex: 3, lastSliceIndex: 2, assembledSize: 200, sliceHashCode: -840689321, lastSliceHashCode: -840689321 01:50:16,15 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76] - expecting more 01:50:16,15 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-668: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76], sliceIndex=3, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251]] 01:50:16,15 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-668: getNextSlice: total size: 448, offset: 300, size: 100, index: 4 01:50:16,15 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-668: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76], data.length=100, sliceIndex=4, totalSlices=5, lastSliceHashCode=-1062253268, replyTo=Actor[akka://cluster-test/user/$a#725556861]] 01:50:16,16 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76], data.length=100, sliceIndex=4, totalSlices=5, lastSliceHashCode=-1062253268, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#725556861]] 01:50:16,16 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76], sliceIndex: 4, lastSliceIndex: 3, assembledSize: 300, sliceHashCode: -1062253268, lastSliceHashCode: -1062253268 01:50:16,16 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76] - expecting more 01:50:16,16 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-668: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76], sliceIndex=4, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251]] 01:50:16,16 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-668: getNextSlice: total size: 448, offset: 400, size: 48, index: 5 01:50:16,16 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-668: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76], data.length=48, sliceIndex=5, totalSlices=5, lastSliceHashCode=248826379, replyTo=Actor[akka://cluster-test/user/$a#725556861]] 01:50:16,17 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76], data.length=48, sliceIndex=5, totalSlices=5, lastSliceHashCode=248826379, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#725556861]] 01:50:16,17 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76], sliceIndex: 5, lastSliceIndex: 4, assembledSize: 400, sliceHashCode: 248826379, lastSliceHashCode: 248826379 01:50:16,17 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Received last slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76] 01:50:16,17 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Removing state for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76] 01:50:16,18 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: AssembledMessageState for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76] was removed from the cache due to EXPLICIT 01:50:16,18 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Notifying callback of re-assembled message Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#725556861], modifications=1}} 01:50:16,18 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testLargeReadReplySlicing: allocating new open transaction member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1 01:50:16,24 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-668: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76], sliceIndex=5, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251]] 01:50:16,24 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-668: Received last slice reply for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76] 01:50:16,24 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-668: Removing state for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76] 01:50:16,24 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-668: SlicedMessageState for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=76] was removed from the cache due to EXPLICIT 01:50:16,24 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#725556861], modifications=1} with Envelope{sessionId=0, txSequence=0, message=ModifyTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0}} 01:50:16,24 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6f0c75c2 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#725556861], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=762318444613} 01:50:16,24 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6f0c75c2 transmitted 1 requests 01:50:16,24 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#725556861], modifications=1}, enqueuedTicks=762318444613, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ModifyTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0}} 01:50:16,24 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#725556861], modifications=1} completed with ModifyTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0} 01:50:16,25 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-667: slice: identifier: member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1}} 01:50:16,25 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-667: Message size: 390 bytes, total slices to send: 4 01:50:16,25 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-667: getNextSlice: total size: 390, offset: 0, size: 100, index: 1 01:50:16,25 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-667: Sending first slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77], data.length=100, sliceIndex=1, totalSlices=4, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251]] 01:50:16,26 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77], data.length=100, sliceIndex=1, totalSlices=4, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251]] 01:50:16,26 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Received first slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77] - creating AssembledMessageState 01:50:16,27 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AssembledMessageState - member-2-frontend-datastore-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77], sliceIndex: 1, lastSliceIndex: 0, assembledSize: 0, sliceHashCode: -1, lastSliceHashCode: -1 01:50:16,27 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77] - expecting more 01:50:16,27 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-670: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77], sliceIndex=1, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#725556861]] 01:50:16,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-667: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77], sliceIndex=1, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#725556861]] 01:50:16,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-667: getNextSlice: total size: 390, offset: 100, size: 100, index: 2 01:50:16,28 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-667: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77], data.length=100, sliceIndex=2, totalSlices=4, lastSliceHashCode=-1569038920, replyTo=Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251]] 01:50:16,29 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77], data.length=100, sliceIndex=2, totalSlices=4, lastSliceHashCode=-1569038920, replyTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251]] 01:50:16,29 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AssembledMessageState - member-2-frontend-datastore-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77], sliceIndex: 2, lastSliceIndex: 1, assembledSize: 100, sliceHashCode: -1569038920, lastSliceHashCode: -1569038920 01:50:16,29 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77] - expecting more 01:50:16,29 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-670: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77], sliceIndex=2, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#725556861]] 01:50:16,29 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-667: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77], sliceIndex=2, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#725556861]] 01:50:16,29 PM [cluster-test-shard-dispatcher-21] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-667: getNextSlice: total size: 390, offset: 200, size: 100, index: 3 01:50:16,29 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-667: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77], data.length=100, sliceIndex=3, totalSlices=4, lastSliceHashCode=1439492931, replyTo=Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251]] 01:50:16,30 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77], data.length=100, sliceIndex=3, totalSlices=4, lastSliceHashCode=1439492931, replyTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251]] 01:50:16,30 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AssembledMessageState - member-2-frontend-datastore-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77], sliceIndex: 3, lastSliceIndex: 2, assembledSize: 200, sliceHashCode: 1439492931, lastSliceHashCode: 1439492931 01:50:16,30 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77] - expecting more 01:50:16,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-670: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77], sliceIndex=3, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#725556861]] 01:50:16,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-667: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77], sliceIndex=3, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#725556861]] 01:50:16,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-667: getNextSlice: total size: 390, offset: 300, size: 90, index: 4 01:50:16,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-667: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77], data.length=90, sliceIndex=4, totalSlices=4, lastSliceHashCode=-31040526, replyTo=Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251]] 01:50:16,32 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77], data.length=90, sliceIndex=4, totalSlices=4, lastSliceHashCode=-31040526, replyTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251]] 01:50:16,32 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AssembledMessageState - member-2-frontend-datastore-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77], sliceIndex: 4, lastSliceIndex: 3, assembledSize: 300, sliceHashCode: -31040526, lastSliceHashCode: -31040526 01:50:16,32 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Received last slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77] 01:50:16,33 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Removing state for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77] 01:50:16,33 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: AssembledMessageState for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77] was removed from the cache due to EXPLICIT 01:50:16,33 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Notifying callback of re-assembled message Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1}} 01:50:16,34 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#725556861], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1}} 01:50:16,34 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#725556861], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=762318444613, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1}} 01:50:16,34 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1} 01:50:16,34 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLargeReadReplySlicing, generation=0} 01:50:16,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Issuing gracefulStop to shard member-1-shard-cars-testLargeReadReplySlicing 01:50:16,34 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLargeReadReplySlicing - waiting on 1 shards 01:50:16,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: onShutDown 01:50:16,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Initiating leader transfer 01:50:16,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@f875524 01:50:16,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLargeReadReplySlicing: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@f875524 01:50:16,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Attempting to transfer leadership 01:50:16,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLargeReadReplySlicing: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:16,35 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLargeReadReplySlicing, leaderId=null, leaderPayloadVersion=13] 01:50:16,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:16,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:16,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-670: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77], sliceIndex=4, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#725556861]] 01:50:16,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-667: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77], sliceIndex=4, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#725556861]] 01:50:16,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-667: Received last slice reply for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77] 01:50:16,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-667: Removing state for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77] 01:50:16,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-667: SlicedMessageState for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=667, messageId=77] was removed from the cache due to EXPLICIT 01:50:16,36 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLargeReadReplySlicing] 01:50:16,36 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing Received follower initial sync status for member-2-shard-cars-testLargeReadReplySlicing status sync done true 01:50:16,36 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLargeReadReplySlicing, leaderId=null, leaderPayloadVersion=13] 01:50:16,36 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:16,36 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:16,36 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6f0c75c2 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@5ae0c9ff, splicing queue 01:50:16,36 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6f0c75c2 queue spliced 0 messages 01:50:16,36 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:16,36 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: refreshing backend for shard 1 01:50:16,36 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testLargeReadReplySlicing, matchIndex: -1, lastIndex: -1, isVoting: true 01:50:16,36 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:50:16,36 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Follower's log matches - sending ElectionTimeout 01:50:16,36 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:16,36 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLargeReadReplySlicing: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:16,36 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:16,36 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-135333977] 01:50:16,36 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Leader transfer complete 01:50:16,36 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:16,36 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testLargeReadReplySlicing, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:50:16,453 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: All shards shutdown - sending PoisonPill to self 01:50:16,453 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] ShardManager - shard-manager-testLargeReadReplySlicing: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:16,454 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-1-shard-cars-testLargeReadReplySlicing 01:50:16,454 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-670: Closing 01:50:16,454 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Aborting 0 pending queued transactions 01:50:16,454 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testLargeReadReplySlicing 01:50:16,454 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLargeReadReplySlicing: Closing 01:50:16,454 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLargeReadReplySlicing, generation=0} 01:50:16,455 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLargeReadReplySlicing: Closing 01:50:16,455 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLargeReadReplySlicing: shutting down 01:50:16,455 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLargeReadReplySlicing: Closing 01:50:16,456 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:16,456 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/11/2024 13:50:16.462] [cluster-test-akka.remote.default-remote-dispatcher-8] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1010)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:50:16.462] [cluster-test-akka.remote.default-remote-dispatcher-8] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1010)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:50:16.462] [cluster-test-akka.remote.default-remote-dispatcher-8] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1010)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:50:16.462] [cluster-test-akka.remote.default-remote-dispatcher-8] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1010)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:16,480 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Closing 01:50:16,481 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testLargeReadReplySlicing 01:50:16,481 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testLargeReadReplySlicing (Follower): Closing 01:50:16,481 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: Aborting 0 pending queued transactions 01:50:16,481 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testLargeReadReplySlicing 01:50:16,484 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:16,484 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:17,472 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [8493403872891738569] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:17,497 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:17,497 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:17,500 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:17.501] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:17,507 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:17,507 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:18,524 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:18.526] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:18,538 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:18,543 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2385780496412558509] 01:50:18,543 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:18,545 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:18,545 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:18,546 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:18,547 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:18,555 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:18,555 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:18,555 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:18,556 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:18,561 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-268615506724660277] 01:50:18,561 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:18,562 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:18,562 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:18,569 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:18,569 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:18,572 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:18,577 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [5411249236758201424] 01:50:18,577 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:18,582 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:18,582 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:18,586 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:18,588 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:18,589 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleTransactionsWritesInQuickSuccession 01:50:18,590 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:18,590 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:18,597 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-508121781] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,598 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 01:50:18,600 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: no initial generation override, starting from 0 01:50:18,600 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:50:18,600 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@57edf9 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@fbc6d24 01:50:18,600 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7b05181d 01:50:18,601 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-testSingleTransactionsWritesInQuickSuccession 01:50:18,601 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Creating local shard: member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:18,601 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, 0, 1710165018601, None)) 01:50:18,601 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:18,601 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:18,601 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165018600,0,0)) 01:50:18,602 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@fbc6d24 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@9e897c3 01:50:18,603 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} started 01:50:18,603 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3cd048f 01:50:18,603 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:18,603 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:18,603 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:18,603 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:18,603 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:18,603 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: updated peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325] 01:50:18,604 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, persistent : true 01:50:18,604 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with journal batch size 1 01:50:18,605 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:18,608 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleTransactionsWritesInQuickSuccession 01:50:18,610 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-1:snapshot-read#273521659] 01:50:18,610 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,610 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#1272643158 created and ready for shard:member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:18,610 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#1272643158] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,610 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-1:snapshot-read#273521659] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,610 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:18,610 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:18,610 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Initializing shard [cars] 01:50:18,610 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is initialized 01:50:18,610 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 01:50:18,610 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 01:50:18,610 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from null to Follower 01:50:18,611 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , registered listener akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession 01:50:18,611 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from null to Follower 01:50:18,620 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1008205799] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,621 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 01:50:18,622 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: no initial generation override, starting from 0 01:50:18,622 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:50:18,622 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testSingleTransactionsWritesInQuickSuccession 01:50:18,622 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1c5d91ef to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2fa78f20 01:50:18,622 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Creating local shard: member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:18,622 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5f85b1d2 01:50:18,622 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, 0, 1710165018622, None)) 01:50:18,623 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165018621,0,0)) 01:50:18,623 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@445654c1 01:50:18,623 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2fa78f20 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6838b2fe 01:50:18,624 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} started 01:50:18,625 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:18,625 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:18,626 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, persistent : true 01:50:18,627 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession#-192985453] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,627 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with journal batch size 1 01:50:18,627 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#-2140128639 created and ready for shard:member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:18,627 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-2:snapshot-read#899179862] 01:50:18,627 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#-2140128639] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,627 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-2:snapshot-read#899179862] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,627 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:18,628 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:18,628 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:18,628 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 01:50:18,628 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 01:50:18,628 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from null to Follower 01:50:18,629 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Initializing shard [cars] 01:50:18,629 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession is initialized 01:50:18,629 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession , registered listener akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession 01:50:18,629 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession from null to Follower 01:50:18,630 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:18,630 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:18,630 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:18,630 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:18,630 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:18,632 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:18,637 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:18,675 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-508121781] 01:50:18,675 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: findLocalShard : cars 01:50:18,675 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325] 01:50:18,886 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:18,886 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Received ElectionTimeout - switching to Candidate 01:50:18,886 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] 01:50:18,886 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:18,886 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 1 01:50:18,887 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:18,887 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Closing 01:50:18,887 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 01:50:18,887 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 01:50:18,887 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from Follower to Candidate 01:50:18,887 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from Follower to Candidate 01:50:19,197 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 01:50:19,197 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,197 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 2 01:50:19,205 PM [cluster-test-shard-dispatcher-26] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:19,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 01:50:19,507 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,507 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 3 01:50:19,557 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:19,557 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:19,558 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:19,558 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,558 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,558 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:19,558 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,558 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,558 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,604 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:19,608 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:19,608 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,608 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,608 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:19,608 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,608 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,766 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 01:50:19,766 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,766 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 4 01:50:19,766 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,767 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:19,767 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Cluster state: [] 01:50:19,767 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:19,767 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:19,767 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=null 01:50:19,767 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:19,767 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,767 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:19,768 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:19,768 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:19,768 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] 01:50:19,768 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:19,769 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:19,769 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: true, hasLeader : true 01:50:19,769 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: became leader with frontend state for [] 01:50:19,769 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from Candidate to Leader 01:50:19,769 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderPayloadVersion=13] 01:50:19,769 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:19,769 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:19,769 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from Candidate to Leader 01:50:19,769 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All Shards are ready - data store testSingleTransactionsWritesInQuickSuccession is ready 01:50:19,769 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Closing 01:50:19,769 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: getLeaderAddress leaderId = member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,769 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderPayloadVersion=13] 01:50:19,769 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:19,769 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:19,770 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All Shards are ready - data store testSingleTransactionsWritesInQuickSuccession is ready 01:50:19,770 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession Received follower initial sync status for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession status sync done false 01:50:19,770 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: -1, nextIndex: 0 01:50:19,770 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:19,781 PM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1512490e 01:50:19,781 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1512490e changed state from IDLE to TX_OPEN 01:50:19,781 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} 01:50:19,781 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:19,781 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:19,781 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1008205799] 01:50:19,781 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:19,781 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} shard 1 01:50:19,781 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:50:19,781 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, primaryVersion=13] 01:50:19,781 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:50:19,781 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, primaryVersion=13] 01:50:19,781 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 01:50:19,782 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@2d338547 01:50:19,782 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1512490e changed state from TX_OPEN to IDLE 01:50:19,782 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1512490e readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:50:19,782 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 01:50:19,782 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@140418c9 01:50:19,782 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@3fb5a100, attempting to connect 01:50:19,783 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} is not yet known 01:50:19,783 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, nanosAgo=9980, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:50:19,784 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:50:19,784 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:50:19,784 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:19,784 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5b669fbe instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@414c53e 01:50:19,784 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:50:19,784 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:50:19,784 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766116633914} 01:50:19,784 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@414c53e 01:50:19,784 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:50:19,784 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} 01:50:19,784 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=2, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} 01:50:19,784 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 01:50:19,785 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2218/0x00000008016a3c10@224531d8 01:50:19,785 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=2, protocol=READY}, enqueuedTicks=766116840508} 01:50:19,785 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=2, protocol=READY}} 01:50:19,785 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 417 bytes, total slices to send: 1 01:50:19,785 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 01:50:19,785 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED} 01:50:19,785 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 1 01:50:19,785 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2258/0x00000008016bac68@73b30e1f 01:50:19,785 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE}, enqueuedTicks=766116929619} 01:50:19,785 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE}} 01:50:19,785 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 297 bytes, total slices to send: 1 01:50:19,785 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 01:50:19,785 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1512490e finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:19,785 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 01:50:19,785 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@4a56caea 01:50:19,785 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@424f20d2 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@60589619, splicing queue 01:50:19,785 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@424f20d2 queue spliced 0 messages 01:50:19,785 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.253 ms 01:50:19,787 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 01:50:19,787 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 01:50:19,787 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:50:19,787 PM [cluster-test-shard-dispatcher-13] [DEBUG] LeaderFrontendState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:50:19,787 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:50:19,787 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:50:19,787 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@180b3178 01:50:19,787 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:50:19,787 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 ready 01:50:19,787 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 01:50:19,787 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 to ready 01:50:19,787 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 is already in state READY (READY) 01:50:19,787 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 initiating direct canCommit 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 validated 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 initiating direct preCommit 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 candidate ready 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1: canCommit - candidate rootPath: / 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1: canCommit - messages: [] 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1: preCommit - successfulFromPrevious: [] 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 prepared 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 initiating direct commit 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 184 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}]]] 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 submitted to persistence 01:50:19,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 01:50:19,789 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 01:50:19,789 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=2, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0}} 01:50:19,789 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=2, protocol=READY}, enqueuedTicks=766116840508, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0}} 01:50:19,789 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=2, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0} 01:50:19,789 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}]]] 01:50:19,789 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 01:50:19,790 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:19,790 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75} 01:50:19,790 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 01:50:19,790 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=184} 01:50:19,790 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 01:50:19,790 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 01:50:19,790 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:19,790 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 01:50:19,790 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 01:50:19,790 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession Received follower initial sync status for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession status sync done true 01:50:19,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 1, nextIndex: 2 01:50:19,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 01:50:19,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 0 01:50:19,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from -1 to 0 01:50:19,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75} 01:50:19,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2220/0x00000008016a65d8@78fca430 01:50:19,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] LeaderFrontendState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:50:19,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@7a1f6e92}, purged=MutableUnsignedLongSet{size=0}} 01:50:19,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:50:19,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 1 01:50:19,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 0 to 1 01:50:19,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 1 data Simple{size=184} 01:50:19,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:50:19,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:50:19,792 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1: commit - successfulFromPrevious: [] 01:50:19,792 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:50:19,792 PM [cluster-test-shard-dispatcher-12] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 01:50:19,792 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#770667477] with name member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher 01:50:19,792 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 0 for term 4 01:50:19,792 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:19,792 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=1 01:50:19,792 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#770667477] 01:50:19,792 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:19,792 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#770667477] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:19,793 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 114.5 μs 01:50:19,793 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 1 01:50:19,793 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 01:50:19,793 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 0 01:50:19,793 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from -1 to 0 01:50:19,793 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75} 01:50:19,793 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@122c43b3}, purged=MutableUnsignedLongSet{size=0}} 01:50:19,793 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:50:19,793 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 1 01:50:19,793 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 0 to 1 01:50:19,793 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 1 data Simple{size=184} 01:50:19,793 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:50:19,793 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:50:19,793 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-1440797528] with name member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher 01:50:19,794 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 0 for term 4 01:50:19,794 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=1 01:50:19,794 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-1440797528] 01:50:19,794 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:19,794 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 65.64 μs 01:50:19,794 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-1440797528] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:19,794 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0}} 01:50:19,794 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766116633914, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0}} 01:50:19,794 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0} 01:50:19,794 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1}} 01:50:19,794 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE}, enqueuedTicks=766116929619, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1}} 01:50:19,794 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE} completed with TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1} 01:50:19,794 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} directCommit completed 01:50:19,794 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: initiating purge 01:50:19,794 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:50:19,794 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@3dca0d9a 01:50:19,794 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1512490e changed state from IDLE to TX_OPEN 01:50:19,794 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} 01:50:19,794 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED}: initiating purge 01:50:19,794 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 2 01:50:19,794 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@68665b3e 01:50:19,794 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:19,794 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766129384485} 01:50:19,794 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:50:19,794 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 0 01:50:19,794 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@170849c6 01:50:19,794 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY}, enqueuedTicks=766129708110} 01:50:19,794 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY}} 01:50:19,795 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 457 bytes, total slices to send: 1 01:50:19,795 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 01:50:19,795 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1512490e changed state from TX_OPEN to IDLE 01:50:19,795 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1512490e readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:50:19,795 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=FLUSHED} allocated sequence 1 01:50:19,795 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@39fd3b4f 01:50:19,795 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE}, enqueuedTicks=766130076856} 01:50:19,795 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE}} 01:50:19,795 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 298 bytes, total slices to send: 1 01:50:19,795 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 01:50:19,798 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:50:19,798 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 01:50:19,798 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 01:50:19,798 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:19,798 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:50:19,798 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:50:19,798 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@3a116b88 01:50:19,798 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:50:19,798 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 ready 01:50:19,798 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}], operation=WRITE}}}}}}}]} 01:50:19,798 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 to ready 01:50:19,798 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 is already in state READY (READY) 01:50:19,798 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 initiating direct canCommit 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 validated 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 initiating direct preCommit 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 candidate ready 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1: canCommit - candidate rootPath: / 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1: canCommit - messages: [] 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1: preCommit - successfulFromPrevious: [] 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 prepared 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 initiating direct commit 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}]]] 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 submitted to persistence 01:50:19,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 01:50:19,800 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 01:50:19,801 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=4, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0}} 01:50:19,801 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY}, enqueuedTicks=766129708110, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0}} 01:50:19,801 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0} 01:50:19,801 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}]]] 01:50:19,801 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 01:50:19,801 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:50:19,801 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76} 01:50:19,801 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 01:50:19,801 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 01:50:19,801 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 01:50:19,801 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 01:50:19,801 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:19,801 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 01:50:19,801 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 01:50:19,801 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 3, nextIndex: 4 01:50:19,801 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 01:50:19,801 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 2 01:50:19,801 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 1 to 2 01:50:19,801 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76} 01:50:19,802 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@1ef37454 01:50:19,802 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:50:19,802 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:50:19,802 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 3 01:50:19,802 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 2 to 3 01:50:19,802 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 3 data Simple{size=241} 01:50:19,802 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:50:19,802 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:50:19,802 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1: commit - successfulFromPrevious: [] 01:50:19,802 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 01:50:19,802 PM [cluster-test-shard-dispatcher-13] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:50:19,802 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 2 for term 4 01:50:19,802 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:50:19,802 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=3 01:50:19,802 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:19,802 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 81.22 μs 01:50:19,803 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 3 01:50:19,803 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 01:50:19,803 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 2 01:50:19,803 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 1 to 2 01:50:19,803 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76} 01:50:19,803 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:50:19,803 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 3 01:50:19,803 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 2 to 3 01:50:19,803 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 3 data Simple{size=241} 01:50:19,803 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:50:19,804 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:50:19,804 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 2 for term 4 01:50:19,804 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=3 01:50:19,804 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2}} 01:50:19,804 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766129384485, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2}} 01:50:19,804 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 01:50:19,804 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@414c53e purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=DONE} 01:50:19,804 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: purge completed 01:50:19,804 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:19,804 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5b669fbe purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 01:50:19,804 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=5, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1}} 01:50:19,804 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE}, enqueuedTicks=766130076856, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1}} 01:50:19,804 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=FLUSHED} directCommit completed 01:50:19,804 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=FLUSHED}: initiating purge 01:50:19,804 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=DONE} allocated sequence 2 01:50:19,804 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 55.80 μs 01:50:19,804 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@52576afc 01:50:19,804 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:19,804 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766139203646} 01:50:19,804 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 01:50:19,804 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 0 01:50:19,804 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@734d77e9 01:50:19,804 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=766139373019} 01:50:19,805 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:50:19,805 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 01:50:19,805 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 01:50:19,805 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:19,805 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 01:50:19,805 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:50:19,805 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@1e5e2395 01:50:19,805 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-672: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0}} 01:50:19,805 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-672: Message size: 388 bytes, total slices to send: 1 01:50:19,805 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-672: Message does not need to be sliced - sending original message 01:50:19,805 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 01:50:19,806 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0}} 01:50:19,806 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=766139373019, sessionId=0, txSequence=7} with Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0}} 01:50:19,806 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0} 01:50:19,806 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@414c53e aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} 01:50:19,806 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 1 01:50:19,807 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@169582d2 01:50:19,807 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT}, enqueuedTicks=766141819389} 01:50:19,807 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT}} 01:50:19,807 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 298 bytes, total slices to send: 1 01:50:19,807 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 01:50:19,807 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1512490e changed state from IDLE to TX_OPEN 01:50:19,807 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:50:19,807 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=OPEN} allocated sequence 0 01:50:19,807 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@15d1c73d 01:50:19,807 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY}, enqueuedTicks=766142266517} 01:50:19,807 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, message: Envelope{sessionId=0, txSequence=9, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY}} 01:50:19,807 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 457 bytes, total slices to send: 1 01:50:19,807 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 01:50:19,807 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1512490e changed state from TX_OPEN to IDLE 01:50:19,807 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1512490e readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:50:19,807 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=FLUSHED} allocated sequence 1 01:50:19,807 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@1127ccf8 01:50:19,807 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE}, enqueuedTicks=766142518941} 01:50:19,807 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, message: Envelope{sessionId=0, txSequence=a, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE}} 01:50:19,807 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 298 bytes, total slices to send: 1 01:50:19,807 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 01:50:19,808 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 01:50:19,808 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 01:50:19,808 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 01:50:19,808 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 01:50:19,809 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 01:50:19,809 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:50:19,809 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:50:19,809 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@1e5e2395 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 ready 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}], operation=WRITE}}}}}}}]} 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 to ready 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 is already in state READY (READY) 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 initiating direct canCommit 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 validated 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 initiating direct preCommit 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 candidate ready 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1: canCommit - candidate rootPath: / 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1: canCommit - messages: [] 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1: preCommit - successfulFromPrevious: [] 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 prepared 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 initiating direct commit 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 4, leaderLastIndex: 6, leaderSnapShotIndex: 2 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}], SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}], SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}]]] 01:50:19,810 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 submitted to persistence 01:50:19,811 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 01:50:19,811 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}], SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}], SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}]]] 01:50:19,811 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 3 01:50:19,811 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 01:50:19,811 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77} 01:50:19,811 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 01:50:19,811 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 01:50:19,811 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 01:50:19,811 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 01:50:19,811 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 01:50:19,811 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 4 01:50:19,811 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:19,811 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0}} 01:50:19,811 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY}, enqueuedTicks=766142266517, sessionId=0, txSequence=9} with Envelope{sessionId=0, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0}} 01:50:19,811 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0} 01:50:19,811 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 01:50:19,811 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 01:50:19,811 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 6, nextIndex: 7 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 3 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 4 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 3 to 4 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77} 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@fb0f78f 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 5 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 4 to 5 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 5 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2264/0x00000008016c74c8@59ad2996 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 6 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 5 to 6 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 6 data Simple{size=241} 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1: commit - successfulFromPrevious: [] 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 changing state from COMMIT_PENDING to COMMITTED 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 5 for term 4 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 01:50:19,812 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=6 01:50:19,812 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:19,813 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 46.20 μs 01:50:19,813 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} with Envelope{sessionId=0, txSequence=6, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2}} 01:50:19,813 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766139203646, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2}} 01:50:19,813 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=DONE}: purge completed 01:50:19,813 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@414c53e purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=DONE} 01:50:19,813 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1}} 01:50:19,813 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT}, enqueuedTicks=766141819389, sessionId=0, txSequence=8} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1}} 01:50:19,813 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1} 01:50:19,813 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN}: initiating purge 01:50:19,813 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=DONE} allocated sequence 2 01:50:19,813 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@70a9adb 01:50:19,813 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766148244355} 01:50:19,813 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 01:50:19,813 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 01:50:19,813 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 01:50:19,813 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:19,814 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 01:50:19,814 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 6 01:50:19,814 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 3, lastIndex(): 6 01:50:19,814 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 4 01:50:19,814 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 3 to 4 01:50:19,814 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77} 01:50:19,814 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:50:19,814 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 5 01:50:19,814 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 4 to 5 01:50:19,814 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 5 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 01:50:19,814 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 01:50:19,814 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 6 01:50:19,814 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 5 to 6 01:50:19,814 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 6 data Simple{size=241} 01:50:19,814 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:50:19,815 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:50:19,815 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 5 for term 4 01:50:19,815 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=6 01:50:19,815 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:19,815 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 35.51 μs 01:50:19,815 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1}} 01:50:19,815 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE}, enqueuedTicks=766142518941, sessionId=0, txSequence=10} with Envelope{sessionId=0, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1}} 01:50:19,815 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=FLUSHED} directCommit completed 01:50:19,815 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=FLUSHED}: initiating purge 01:50:19,815 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=DONE} allocated sequence 2 01:50:19,815 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@227bc6bd 01:50:19,815 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766150184967} 01:50:19,817 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 01:50:19,817 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} allocated sequence 0 01:50:19,817 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@1dd5a28e 01:50:19,817 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=766152215630} 01:50:19,817 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 7 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,817 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}]]] 01:50:19,821 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:50:19,821 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 01:50:19,821 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 01:50:19,821 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, logIndex: 8, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:19,821 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 01:50:19,821 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:50:19,821 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@725ab2f0 01:50:19,822 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-672: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, message: Envelope{sessionId=0, txSequence=d, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0}} 01:50:19,822 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-672: Message size: 408 bytes, total slices to send: 1 01:50:19,822 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-672: Message does not need to be sliced - sending original message 01:50:19,822 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 01:50:19,823 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=d, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0}} 01:50:19,823 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=766152215630, sessionId=0, txSequence=13} with Envelope{sessionId=0, txSequence=d, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0}} 01:50:19,823 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0} 01:50:19,823 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}]]] 01:50:19,823 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:50:19,823 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 01:50:19,823 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 01:50:19,823 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@414c53e aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} 01:50:19,823 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 01:50:19,823 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} allocated sequence 1 01:50:19,823 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@22109a27 01:50:19,823 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 01:50:19,823 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT}, enqueuedTicks=766158225939} 01:50:19,823 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, message: Envelope{sessionId=0, txSequence=e, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT}} 01:50:19,823 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=7, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:19,823 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 298 bytes, total slices to send: 1 01:50:19,823 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 01:50:19,823 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1512490e changed state from IDLE to TX_OPEN 01:50:19,823 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:50:19,823 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=OPEN} allocated sequence 0 01:50:19,823 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@5c160282 01:50:19,823 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY}, enqueuedTicks=766158606905} 01:50:19,823 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, message: Envelope{sessionId=0, txSequence=f, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY}} 01:50:19,823 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 01:50:19,823 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 457 bytes, total slices to send: 1 01:50:19,823 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 01:50:19,824 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1512490e changed state from TX_OPEN to IDLE 01:50:19,824 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1512490e readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:50:19,824 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=FLUSHED} allocated sequence 1 01:50:19,824 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@695da159 01:50:19,824 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE}, enqueuedTicks=766158864949} 01:50:19,824 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, message: Envelope{sessionId=0, txSequence=10, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE}} 01:50:19,824 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 298 bytes, total slices to send: 1 01:50:19,824 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 01:50:19,824 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 7, nextIndex: 8 01:50:19,824 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 6 01:50:19,824 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 7 01:50:19,824 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 6 to 7 01:50:19,824 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 7 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 01:50:19,824 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@464c392f 01:50:19,824 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 01:50:19,824 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 01:50:19,824 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 6 for term 4 01:50:19,824 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 8, leaderLastIndex: 8, leaderSnapShotIndex: 6 01:50:19,824 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 8 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,824 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=7, prevLogTerm=4, leaderCommit=7, replicatedToAllIndex=6, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}]]] 01:50:19,824 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=7 01:50:19,824 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 01:50:19,824 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 01:50:19,824 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 01:50:19,824 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, logIndex: 9, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 01:50:19,826 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:50:19,826 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:50:19,826 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@725ab2f0 01:50:19,826 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:50:19,826 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 ready 01:50:19,826 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}], operation=WRITE}}}}}}}]} 01:50:19,826 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 to ready 01:50:19,826 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 is already in state READY (READY) 01:50:19,826 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 initiating direct canCommit 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 validated 01:50:19,827 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=7, prevLogTerm=4, leaderCommit=7, replicatedToAllIndex=6, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}]]] 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:19,827 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 initiating direct preCommit 01:50:19,827 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 7, entries to be added from: 0 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:50:19,827 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77} 01:50:19,827 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 01:50:19,827 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 01:50:19,827 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 7 01:50:19,827 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=8, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 candidate ready 01:50:19,827 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 7, context.getLastApplied(): 6, lastIndex(): 8 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1: canCommit - candidate rootPath: / 01:50:19,827 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 7 01:50:19,827 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 6 to 7 01:50:19,827 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 7 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1: canCommit - messages: [] 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1: preCommit - successfulFromPrevious: [] 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 prepared 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 initiating direct commit 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:50:19,827 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 01:50:19,827 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 6 for term 4 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, logIndex: 10, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 8, leaderLastIndex: 10, leaderSnapShotIndex: 6 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 8 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 submitted to persistence 01:50:19,827 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} with Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2}} 01:50:19,827 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766148244355, sessionId=0, txSequence=11} with Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2}} 01:50:19,827 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=DONE}: purge completed 01:50:19,827 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@414c53e purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=DONE} 01:50:19,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 01:50:19,828 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 01:50:19,828 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=7 01:50:19,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 8, nextIndex: 9 01:50:19,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 8, lastAppliedIndex: 7 01:50:19,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 8 01:50:19,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 7 to 8 01:50:19,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 8 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77} 01:50:19,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@247fc52 01:50:19,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:50:19,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:50:19,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 7 for term 4 01:50:19,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 9, leaderLastIndex: 10, leaderSnapShotIndex: 7 01:50:19,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 9 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=8, prevLogTerm=4, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}], SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}]]] 01:50:19,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=8 01:50:19,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 01:50:19,828 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=f, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0}} 01:50:19,828 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY}, enqueuedTicks=766158606905, sessionId=0, txSequence=15} with Envelope{sessionId=0, txSequence=f, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0}} 01:50:19,828 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0} 01:50:19,829 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=8, prevLogTerm=4, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}], SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}]]] 01:50:19,829 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} with Envelope{sessionId=0, txSequence=c, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2}} 01:50:19,829 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 01:50:19,829 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 8, entries to be added from: 0 01:50:19,829 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766150184967, sessionId=0, txSequence=12} with Envelope{sessionId=0, txSequence=c, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2}} 01:50:19,829 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 01:50:19,829 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=DONE}: purge completed 01:50:19,829 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 01:50:19,829 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@414c53e purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=DONE} 01:50:19,829 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 01:50:19,829 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 01:50:19,829 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 4 01:50:19,829 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 8 01:50:19,829 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=10, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:19,829 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 8, context.getLastApplied(): 7, lastIndex(): 10 01:50:19,829 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 8 01:50:19,829 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 7 to 8 01:50:19,829 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 8 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77} 01:50:19,829 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:50:19,829 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 7 for term 4 01:50:19,830 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 01:50:19,830 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 01:50:19,830 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=8 01:50:19,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 10, nextIndex: 11 01:50:19,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 10, lastAppliedIndex: 8 01:50:19,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 9 01:50:19,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 8 to 9 01:50:19,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 9 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 01:50:19,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2264/0x00000008016c74c8@8147d5c 01:50:19,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 01:50:19,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 10 01:50:19,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 9 to 10 01:50:19,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 10 data Simple{size=241} 01:50:19,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:50:19,831 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:50:19,831 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1: commit - successfulFromPrevious: [] 01:50:19,831 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 changing state from COMMIT_PENDING to COMMITTED 01:50:19,831 PM [cluster-test-shard-dispatcher-12] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:50:19,831 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:19,831 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 9 for term 4 01:50:19,831 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 61.99 μs 01:50:19,831 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 11, leaderLastIndex: 10, leaderSnapShotIndex: 9 01:50:19,831 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=10 01:50:19,832 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 10 01:50:19,832 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=e, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1}} 01:50:19,832 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 10, context.getLastApplied(): 8, lastIndex(): 10 01:50:19,832 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 9 01:50:19,832 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 8 to 9 01:50:19,832 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT}, enqueuedTicks=766158225939, sessionId=0, txSequence=14} with Envelope{sessionId=0, txSequence=e, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1}} 01:50:19,832 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 9 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 01:50:19,832 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1} 01:50:19,832 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN}: initiating purge 01:50:19,832 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=DONE} allocated sequence 2 01:50:19,832 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 01:50:19,832 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 10 01:50:19,832 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 9 to 10 01:50:19,832 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@4f30b9b5 01:50:19,832 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 10 data Simple{size=241} 01:50:19,832 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766167561273} 01:50:19,832 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=10, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1}} 01:50:19,832 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE}, enqueuedTicks=766158864949, sessionId=0, txSequence=16} with Envelope{sessionId=0, txSequence=10, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1}} 01:50:19,832 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=FLUSHED} directCommit completed 01:50:19,832 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=FLUSHED}: initiating purge 01:50:19,832 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=DONE} allocated sequence 2 01:50:19,832 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@15d5b4c6 01:50:19,832 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:50:19,832 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766167724166} 01:50:19,833 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 01:50:19,833 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} allocated sequence 0 01:50:19,833 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@5e02d1c3 01:50:19,833 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=766167993880} 01:50:19,833 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:50:19,833 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:19,833 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 9 for term 4 01:50:19,833 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 41.66 μs 01:50:19,833 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=10 01:50:19,833 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 01:50:19,834 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 01:50:19,834 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 01:50:19,834 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, logIndex: 11, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:19,834 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:50:19,834 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 01:50:19,834 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 01:50:19,834 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, logIndex: 12, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:19,834 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 01:50:19,834 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:50:19,834 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@46e551d4 01:50:19,834 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-672: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, message: Envelope{sessionId=0, txSequence=13, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0}} 01:50:19,834 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 01:50:19,834 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-672: Message size: 431 bytes, total slices to send: 1 01:50:19,834 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-672: Message does not need to be sliced - sending original message 01:50:19,835 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 01:50:19,836 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=13, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0}} 01:50:19,836 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=766167993880, sessionId=0, txSequence=19} with Envelope{sessionId=0, txSequence=13, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0}} 01:50:19,836 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0} 01:50:19,836 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@414c53e aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} 01:50:19,836 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} allocated sequence 1 01:50:19,836 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@3d4e1185 01:50:19,836 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT}, enqueuedTicks=766170993860} 01:50:19,836 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, message: Envelope{sessionId=0, txSequence=14, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT}} 01:50:19,836 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 298 bytes, total slices to send: 1 01:50:19,836 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 01:50:19,836 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1512490e changed state from IDLE to TX_OPEN 01:50:19,836 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:50:19,836 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=OPEN} allocated sequence 0 01:50:19,836 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@2167ebd3 01:50:19,836 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY}, enqueuedTicks=766171372036} 01:50:19,836 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, message: Envelope{sessionId=0, txSequence=15, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY}} 01:50:19,836 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 457 bytes, total slices to send: 1 01:50:19,836 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 01:50:19,836 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1512490e changed state from TX_OPEN to IDLE 01:50:19,836 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1512490e readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:50:19,836 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=FLUSHED} allocated sequence 1 01:50:19,836 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@691e2b58 01:50:19,836 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE}, enqueuedTicks=766171593189} 01:50:19,836 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, message: Envelope{sessionId=0, txSequence=16, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE}} 01:50:19,836 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 298 bytes, total slices to send: 1 01:50:19,836 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, logIndex: 13, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@46e551d4 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 ready 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}], operation=WRITE}}}}}}}]} 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 to ready 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 is already in state READY (READY) 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 initiating direct canCommit 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 validated 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 initiating direct preCommit 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 candidate ready 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1: canCommit - candidate rootPath: / 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1: canCommit - messages: [] 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1: preCommit - successfulFromPrevious: [] 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 prepared 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 initiating direct commit 01:50:19,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:50:19,838 PM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 01:50:19,838 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 01:50:19,838 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 01:50:19,838 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, logIndex: 14, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:19,838 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 11, leaderLastIndex: 14, leaderSnapShotIndex: 9 01:50:19,838 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 11 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,838 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=10, prevLogTerm=4, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}], SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}], SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}], SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}]]] 01:50:19,838 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 submitted to persistence 01:50:19,838 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 01:50:19,838 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 01:50:19,841 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=15, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0}} 01:50:19,841 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY}, enqueuedTicks=766171372036, sessionId=0, txSequence=21} with Envelope{sessionId=0, txSequence=15, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0}} 01:50:19,841 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0} 01:50:19,842 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=10, prevLogTerm=4, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}], SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}], SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}], SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}]]] 01:50:19,842 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 4 01:50:19,842 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 10, entries to be added from: 0 01:50:19,842 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 01:50:19,842 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 01:50:19,842 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77} 01:50:19,842 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 01:50:19,842 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 01:50:19,842 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 01:50:19,842 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 01:50:19,842 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 01:50:19,842 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 5 01:50:19,842 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=14, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:19,842 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 01:50:19,842 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 01:50:19,842 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 01:50:19,842 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 14, nextIndex: 15 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 14, lastAppliedIndex: 10 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 11 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 10 to 11 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 11 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@4793035f 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 12 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 11 to 12 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 12 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77} 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@6e67a323 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 13 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 12 to 13 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 13 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2264/0x00000008016c74c8@4cf42a3c 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 14 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 13 to 14 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 14 data Simple{size=241} 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1: commit - successfulFromPrevious: [] 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 changing state from COMMIT_PENDING to COMMITTED 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 13 for term 4 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 15, leaderLastIndex: 14, leaderSnapShotIndex: 13 01:50:19,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=14 01:50:19,843 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:19,843 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 47.11 μs 01:50:19,844 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 14 01:50:19,844 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 14, context.getLastApplied(): 10, lastIndex(): 14 01:50:19,844 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 11 01:50:19,844 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 10 to 11 01:50:19,844 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 11 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} with Envelope{sessionId=0, txSequence=11, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2}} 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766167561273, sessionId=0, txSequence=17} with Envelope{sessionId=0, txSequence=11, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2}} 01:50:19,844 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=DONE}: purge completed 01:50:19,844 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 12 01:50:19,844 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 11 to 12 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@414c53e purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=DONE} 01:50:19,844 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 12 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77} 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} with Envelope{sessionId=0, txSequence=12, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2}} 01:50:19,844 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:50:19,844 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 13 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766167724166, sessionId=0, txSequence=18} with Envelope{sessionId=0, txSequence=12, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2}} 01:50:19,844 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 12 to 13 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=DONE}: purge completed 01:50:19,844 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 13 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@414c53e purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=DONE} 01:50:19,844 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 01:50:19,844 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 14 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=14, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1}} 01:50:19,844 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 13 to 14 01:50:19,844 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 14 data Simple{size=241} 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT}, enqueuedTicks=766170993860, sessionId=0, txSequence=20} with Envelope{sessionId=0, txSequence=14, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1}} 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1} 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN}: initiating purge 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=DONE} allocated sequence 2 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@465a38e5 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766179423698} 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=16, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1}} 01:50:19,844 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE}, enqueuedTicks=766171593189, sessionId=0, txSequence=22} with Envelope{sessionId=0, txSequence=16, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1}} 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=FLUSHED} directCommit completed 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=FLUSHED}: initiating purge 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=DONE} allocated sequence 2 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@3ae840bd 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766179562010} 01:50:19,844 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:50:19,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:19,844 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 13 for term 4 01:50:19,845 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 37.70 μs 01:50:19,845 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=14 01:50:19,845 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 01:50:19,845 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} allocated sequence 0 01:50:19,845 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@72ab2055 01:50:19,845 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=766180018618} 01:50:19,845 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 01:50:19,845 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 01:50:19,845 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 01:50:19,845 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, logIndex: 15, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:19,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:50:19,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 01:50:19,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 01:50:19,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, logIndex: 16, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:19,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 01:50:19,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:50:19,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@b5a0e8a 01:50:19,846 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-672: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, message: Envelope{sessionId=0, txSequence=19, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0}} 01:50:19,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 01:50:19,846 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-672: Message size: 451 bytes, total slices to send: 1 01:50:19,846 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-672: Message does not need to be sliced - sending original message 01:50:19,846 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 01:50:19,847 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=19, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0}} 01:50:19,847 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=766180018618, sessionId=0, txSequence=25} with Envelope{sessionId=0, txSequence=19, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0}} 01:50:19,847 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0} 01:50:19,847 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@414c53e aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} 01:50:19,847 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} allocated sequence 1 01:50:19,847 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@45cb7e8a 01:50:19,847 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT}, enqueuedTicks=766182636741} 01:50:19,847 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, message: Envelope{sessionId=0, txSequence=1a, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT}} 01:50:19,847 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 298 bytes, total slices to send: 1 01:50:19,847 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 01:50:19,848 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1512490e changed state from IDLE to TX_OPEN 01:50:19,848 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:50:19,848 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=OPEN} allocated sequence 0 01:50:19,848 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@3fa69ae5 01:50:19,848 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY}, enqueuedTicks=766182986137} 01:50:19,848 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, message: Envelope{sessionId=0, txSequence=1b, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY}} 01:50:19,848 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 457 bytes, total slices to send: 1 01:50:19,848 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 01:50:19,848 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1512490e changed state from TX_OPEN to IDLE 01:50:19,848 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1512490e readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:50:19,848 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=FLUSHED} allocated sequence 1 01:50:19,848 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@76400686 01:50:19,848 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE}, enqueuedTicks=766183222421} 01:50:19,848 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, message: Envelope{sessionId=0, txSequence=1c, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE}} 01:50:19,848 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 298 bytes, total slices to send: 1 01:50:19,848 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 01:50:19,848 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 01:50:19,848 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 01:50:19,848 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, logIndex: 17, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@b5a0e8a 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 ready 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}], operation=WRITE}}}}}}}]} 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 to ready 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 is already in state READY (READY) 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 initiating direct canCommit 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 validated 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 initiating direct preCommit 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 candidate ready 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1: canCommit - candidate rootPath: / 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1: canCommit - messages: [] 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1: preCommit - successfulFromPrevious: [] 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 prepared 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 initiating direct commit 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, logIndex: 18, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 15, leaderLastIndex: 18, leaderSnapShotIndex: 13 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 15 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=14, prevLogTerm=4, leaderCommit=14, replicatedToAllIndex=13, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}], SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}], SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}], SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}]]] 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 submitted to persistence 01:50:19,849 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 01:50:19,850 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=1b, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0}} 01:50:19,850 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY}, enqueuedTicks=766182986137, sessionId=0, txSequence=27} with Envelope{sessionId=0, txSequence=1b, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0}} 01:50:19,850 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0} 01:50:19,850 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 01:50:19,850 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=14, prevLogTerm=4, leaderCommit=14, replicatedToAllIndex=13, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}], SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}], SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}], SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}]]] 01:50:19,850 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 4 01:50:19,850 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 14, entries to be added from: 0 01:50:19,850 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 01:50:19,850 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 01:50:19,850 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77} 01:50:19,850 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 01:50:19,850 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 01:50:19,850 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 01:50:19,850 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 01:50:19,850 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 01:50:19,850 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 5 01:50:19,850 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=18, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:19,851 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 01:50:19,851 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 01:50:19,851 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 01:50:19,851 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 18, nextIndex: 19 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 18, lastAppliedIndex: 14 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 15 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 14 to 15 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 15 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@467e91ab 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 16 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 15 to 16 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 16 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77} 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@20911801 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 17 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 16 to 17 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 17 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2264/0x00000008016c74c8@2043148c 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 18 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 17 to 18 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 18 data Simple{size=241} 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1: commit - successfulFromPrevious: [] 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 changing state from COMMIT_PENDING to COMMITTED 01:50:19,851 PM [cluster-test-shard-dispatcher-13] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:50:19,852 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 17 for term 4 01:50:19,852 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 19, leaderLastIndex: 18, leaderSnapShotIndex: 17 01:50:19,852 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=18 01:50:19,852 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:19,852 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 51.74 μs 01:50:19,852 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} with Envelope{sessionId=0, txSequence=17, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2}} 01:50:19,852 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766179423698, sessionId=0, txSequence=23} with Envelope{sessionId=0, txSequence=17, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2}} 01:50:19,852 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=DONE}: purge completed 01:50:19,852 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@414c53e purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=DONE} 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 18 01:50:19,854 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} with Envelope{sessionId=0, txSequence=18, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2}} 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 18, context.getLastApplied(): 14, lastIndex(): 18 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 15 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 14 to 15 01:50:19,854 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766179562010, sessionId=0, txSequence=24} with Envelope{sessionId=0, txSequence=18, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2}} 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 15 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 01:50:19,854 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=DONE}: purge completed 01:50:19,854 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@414c53e purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=DONE} 01:50:19,854 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=1a, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1}} 01:50:19,854 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT}, enqueuedTicks=766182636741, sessionId=0, txSequence=26} with Envelope{sessionId=0, txSequence=1a, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1}} 01:50:19,854 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1} 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 01:50:19,854 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN}: initiating purge 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 16 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 15 to 16 01:50:19,854 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=DONE} allocated sequence 2 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 16 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77} 01:50:19,854 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@705fab44 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 17 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 16 to 17 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 17 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 01:50:19,854 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766189059816} 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 18 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 17 to 18 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 18 data Simple{size=241} 01:50:19,854 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=1c, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1}} 01:50:19,854 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=SIMPLE}, enqueuedTicks=766183222421, sessionId=0, txSequence=28} with Envelope{sessionId=0, txSequence=1c, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1}} 01:50:19,854 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=FLUSHED} directCommit completed 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:50:19,854 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=FLUSHED}: initiating purge 01:50:19,854 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=DONE} allocated sequence 2 01:50:19,854 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@4814520d 01:50:19,854 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766189219849} 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:50:19,854 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 17 for term 4 01:50:19,854 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=18 01:50:19,854 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 38.32 μs 01:50:19,854 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 01:50:19,854 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} allocated sequence 0 01:50:19,854 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@455b4663 01:50:19,854 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=766189641296} 01:50:19,855 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 01:50:19,855 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 01:50:19,855 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 01:50:19,855 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, logIndex: 19, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:19,855 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:50:19,856 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 01:50:19,856 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 01:50:19,856 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, logIndex: 20, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:19,856 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 01:50:19,856 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:50:19,856 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@3951208a 01:50:19,856 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-672: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, message: Envelope{sessionId=0, txSequence=1f, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0}} 01:50:19,856 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-672: Message size: 471 bytes, total slices to send: 1 01:50:19,856 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-672: Message does not need to be sliced - sending original message 01:50:19,856 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 01:50:19,856 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 01:50:19,857 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=1f, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0}} 01:50:19,857 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=766189641296, sessionId=0, txSequence=31} with Envelope{sessionId=0, txSequence=1f, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0}} 01:50:19,857 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0} 01:50:19,857 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@414c53e aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} 01:50:19,857 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} allocated sequence 1 01:50:19,857 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@16148dc4 01:50:19,857 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT}, enqueuedTicks=766192434412} 01:50:19,857 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, message: Envelope{sessionId=0, txSequence=20, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT}} 01:50:19,857 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 298 bytes, total slices to send: 1 01:50:19,857 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 01:50:19,858 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 01:50:19,858 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 01:50:19,858 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 01:50:19,858 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, logIndex: 21, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 01:50:19,858 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 01:50:20,7 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 19 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:20,7 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=18, prevLogTerm=4, leaderCommit=18, replicatedToAllIndex=17, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}], SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}], SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}]]] 01:50:20,8 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=18, prevLogTerm=4, leaderCommit=18, replicatedToAllIndex=17, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}], SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}], SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}]]] 01:50:20,8 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 3 01:50:20,8 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 18, entries to be added from: 0 01:50:20,8 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 01:50:20,8 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 01:50:20,8 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77} 01:50:20,8 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 01:50:20,8 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 01:50:20,8 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 01:50:20,8 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 4 01:50:20,9 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=21, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:20,9 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 01:50:20,9 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 01:50:20,9 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 21, nextIndex: 22 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 21, lastAppliedIndex: 18 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 19 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 18 to 19 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 19 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@770983bf 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 20 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 19 to 20 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 20 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77} 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@25003c6c 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 21 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 20 to 21 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 21 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2264/0x00000008016c74c8@201bc087 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 20 for term 4 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 22, leaderLastIndex: 21, leaderSnapShotIndex: 20 01:50:20,10 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=21 01:50:20,12 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 21 01:50:20,12 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 21, context.getLastApplied(): 18, lastIndex(): 21 01:50:20,12 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 19 01:50:20,12 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 18 to 19 01:50:20,12 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 19 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 01:50:20,12 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 01:50:20,12 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 20 01:50:20,12 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 19 to 20 01:50:20,12 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} with Envelope{sessionId=0, txSequence=1d, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2}} 01:50:20,12 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 20 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77} 01:50:20,12 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766189059816, sessionId=0, txSequence=29} with Envelope{sessionId=0, txSequence=1d, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2}} 01:50:20,12 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=DONE}: purge completed 01:50:20,12 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:50:20,12 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 21 01:50:20,12 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@414c53e purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=DONE} 01:50:20,12 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 20 to 21 01:50:20,12 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 21 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 01:50:20,12 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 01:50:20,12 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} with Envelope{sessionId=0, txSequence=1e, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2}} 01:50:20,12 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 20 for term 4 01:50:20,12 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766189219849, sessionId=0, txSequence=30} with Envelope{sessionId=0, txSequence=1e, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2}} 01:50:20,12 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=DONE}: purge completed 01:50:20,12 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=21 01:50:20,12 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@414c53e purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=DONE} 01:50:20,12 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=20, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1}} 01:50:20,12 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT}, enqueuedTicks=766192434412, sessionId=0, txSequence=32} with Envelope{sessionId=0, txSequence=20, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1}} 01:50:20,12 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1} 01:50:20,12 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN}: initiating purge 01:50:20,12 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=DONE} allocated sequence 2 01:50:20,12 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@5f9ed6fa 01:50:20,12 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:20,12 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766347491874} 01:50:20,14 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 01:50:20,14 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 01:50:20,14 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 01:50:20,14 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, logIndex: 22, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:20,15 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 01:50:20,127 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 22 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:20,127 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=21, prevLogTerm=4, leaderCommit=21, replicatedToAllIndex=20, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}]]] 01:50:20,128 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=21, prevLogTerm=4, leaderCommit=21, replicatedToAllIndex=20, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}]]] 01:50:20,128 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:50:20,128 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 21, entries to be added from: 0 01:50:20,128 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 01:50:20,128 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 01:50:20,128 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 01:50:20,128 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=22, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:20,128 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 01:50:20,129 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 22, nextIndex: 23 01:50:20,129 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 22, lastAppliedIndex: 21 01:50:20,129 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 22 01:50:20,129 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 21 to 22 01:50:20,129 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 22 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 01:50:20,129 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@233a3c9d 01:50:20,129 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 01:50:20,129 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 01:50:20,129 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 21 for term 4 01:50:20,129 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 23, leaderLastIndex: 22, leaderSnapShotIndex: 21 01:50:20,129 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=22 01:50:20,130 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 22 01:50:20,130 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 22, context.getLastApplied(): 21, lastIndex(): 22 01:50:20,130 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]} with Envelope{sessionId=0, txSequence=21, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2}} 01:50:20,130 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 22 01:50:20,130 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 21 to 22 01:50:20,130 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 22 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 01:50:20,130 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2033570007]}, enqueuedTicks=766347491874, sessionId=0, txSequence=33} with Envelope{sessionId=0, txSequence=21, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2}} 01:50:20,130 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=DONE}: purge completed 01:50:20,130 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@414c53e purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=DONE} 01:50:20,130 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 01:50:20,130 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 21 for term 4 01:50:20,130 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=22 01:50:20,358 PM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-508121781] 01:50:20,358 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: findLocalShard : cars 01:50:20,358 PM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325] 01:50:20,358 PM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.TestShard$RequestFrontendMetadata to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325] 01:50:20,360 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 01:50:20,360 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} allocated sequence 0 01:50:20,360 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@40a7bf4a 01:50:20,360 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:20,360 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=766695535071} 01:50:20,361 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1 01:50:20,361 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:50:20,361 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@3951208a 01:50:20,362 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-672: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, message: Envelope{sessionId=0, txSequence=22, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0}} 01:50:20,362 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-672: Message size: 460 bytes, total slices to send: 1 01:50:20,362 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-672: Message does not need to be sliced - sending original message 01:50:20,364 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=22, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0}} 01:50:20,364 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=766695535071, sessionId=0, txSequence=34} with Envelope{sessionId=0, txSequence=22, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0}} 01:50:20,364 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0} 01:50:20,364 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@414c53e aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} 01:50:20,364 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} allocated sequence 1 01:50:20,364 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@4f505371 01:50:20,364 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:20,364 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT}, enqueuedTicks=766699609448} 01:50:20,364 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, message: Envelope{sessionId=0, txSequence=23, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2033570007], modifications=0, protocol=ABORT}} 01:50:20,365 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 298 bytes, total slices to send: 1 01:50:20,365 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 01:50:20,365 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:50:20,365 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Issuing gracefulStop to shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:20,365 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession - waiting on 1 shards 01:50:20,365 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onShutDown 01:50:20,365 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Initiating leader transfer 01:50:20,365 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=null, leaderPayloadVersion=13] 01:50:20,365 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:20,365 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:20,366 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5dde79db 01:50:20,366 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:50:20,366 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5dde79db 01:50:20,366 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Attempting to transfer leadership 01:50:20,366 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 23, leaderLastIndex: 22, leaderSnapShotIndex: 21 01:50:20,366 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: not currently active leader, rejecting request Envelope{sessionId=0, txSequence=23, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-2033570007], modifications=0, protocol=ABORT}}. isLeader: true, isLeaderActive: false,isLeadershipTransferInProgress: true, paused: true 01:50:20,367 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: request Envelope{sessionId=0, txSequence=23, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-2033570007], modifications=0, protocol=ABORT}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:20,367 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession] 01:50:20,367 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=null, leaderPayloadVersion=13] 01:50:20,367 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:20,367 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:20,367 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@2e2afbc6, splicing queue 01:50:20,367 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:50:20,367 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a4c7ddd queue spliced 1 messages 01:50:20,367 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:20,367 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: refreshing backend for shard 1 01:50:20,367 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:50:20,367 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:20,367 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:20,367 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1008205799] 01:50:20,368 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:20,368 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:50:20,369 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, matchIndex: 22, lastIndex: 22, isVoting: true 01:50:20,369 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Follower's log matches - sending ElectionTimeout 01:50:20,369 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 23, leaderLastIndex: 22, leaderSnapShotIndex: 21 01:50:20,369 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Leader transfer complete 01:50:20,577 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:50:20,776 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All shards shutdown - sending PoisonPill to self 01:50:20,776 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-715270325]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:20,776 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:20,776 PM [cluster-test-shard-dispatcher-13] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader)_slicer-id-675: Closing 01:50:20,776 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborting 0 pending queued transactions 01:50:20,777 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 01:50:20,777 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 01:50:20,777 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:50:20,777 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 01:50:20,777 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: shutting down 01:50:20,777 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 01:50:20,778 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:20,778 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:20,783 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:20,784 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:20,784 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:20,784 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:20,797 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 01:50:20,798 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:20,798 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Closing 01:50:20,798 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborting 0 pending queued transactions 01:50:20,798 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 01:50:20,800 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:20,800 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:21,792 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2385780496412558509] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:21,812 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-268615506724660277] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:21,819 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:21,819 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:21,819 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:21.820] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:21,827 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:21,828 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:21,829 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:21.830] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:21,851 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:21,855 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [5356127688951311061] 01:50:21,856 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:21,857 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:21,857 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:21,861 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:21,861 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:21,867 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:21,867 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:21,867 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:21,872 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:21,883 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [6038874122084283199] 01:50:21,883 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:21,891 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:21,891 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:21,892 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:21,892 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:21,913 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:21,927 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [7867386542107341553] 01:50:21,927 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:21,929 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:21,929 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:21,940 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:21,941 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:21,942 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 01:50:21,944 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:21,944 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:21,960 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:21,966 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1643686012] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:21,966 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:21,967 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:50:21,972 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 01:50:21,972 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 01:50:21,972 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:50:21,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:21,972 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6ac75c60 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4e7149e7 01:50:21,972 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3f37060d 01:50:21,972 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:21,972 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:21,972 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1710165021972, None)) 01:50:21,972 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165021971,0,0)) 01:50:21,972 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4e7149e7 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6195fc52 01:50:21,972 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 01:50:21,973 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:21,974 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:21,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7a706cf1 01:50:21,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:21,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:21,976 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 01:50:21,989 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:21,990 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:21,990 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:21,990 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542] 01:50:21,991 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 01:50:21,991 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:21,991 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-1638977163 created and ready for shard:member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:21,991 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-1638977163] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:21,991 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-1:snapshot-read#381862961] 01:50:21,991 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-1:snapshot-read#381862961] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:21,991 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 01:50:21,991 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:21,993 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:21,993 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:21,993 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 01:50:21,993 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 01:50:21,993 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:50:21,993 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:50:21,993 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 01:50:21,993 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 01:50:21,993 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 01:50:21,995 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1376532396] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:21,997 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:50:22,12 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 01:50:22,12 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:50:22,12 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3ec17f51 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6d887398 01:50:22,12 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@cc94252 01:50:22,12 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1710165022012, None)) 01:50:22,12 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 01:50:22,12 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,12 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:22,12 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,12 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165022011,0,0)) 01:50:22,13 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6d887398 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6b9cd76b 01:50:22,13 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 01:50:22,13 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3755db76 01:50:22,13 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:22,13 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,13 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,13 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,13 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop#1308364278] 01:50:22,14 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 01:50:22,14 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop#1308364278] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:22,14 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#1498917781 created and ready for shard:member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,14 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-2:snapshot-read#1302228546] 01:50:22,14 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#1498917781] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:22,14 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-2:snapshot-read#1302228546] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:22,14 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 01:50:22,15 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:22,15 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:22,15 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:50:22,15 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:50:22,15 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 01:50:22,15 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 01:50:22,15 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,15 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 01:50:22,15 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 01:50:22,15 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 01:50:22,63 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1643686012] 01:50:22,63 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: findLocalShard : cars 01:50:22,63 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542] 01:50:22,227 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:22,227 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Received ElectionTimeout - switching to Candidate 01:50:22,227 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:50:22,227 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=1, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,227 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 1 01:50:22,227 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:22,227 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 01:50:22,227 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:50:22,227 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:50:22,228 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Follower to Candidate 01:50:22,228 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Follower to Candidate 01:50:22,467 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 01:50:22,467 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=2, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,467 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 2 01:50:22,557 PM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:22,777 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 01:50:22,777 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=3, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,777 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 3 01:50:22,877 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:22,877 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:22,878 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:22,878 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,878 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,878 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:22,878 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,878 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,878 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,878 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,882 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:22,883 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,883 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,883 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:22,883 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,883 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,883 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:22,948 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:23,77 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 01:50:23,77 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,77 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 4 01:50:23,77 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,77 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,78 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:23,78 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 01:50:23,78 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:23,78 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:23,78 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=null 01:50:23,78 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,78 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:23,78 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,78 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:23,79 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:23,79 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:23,79 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:50:23,79 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:23,79 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:23,79 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:23,79 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 01:50:23,80 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: became leader with frontend state for [] 01:50:23,80 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Candidate to Leader 01:50:23,80 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 01:50:23,80 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:23,80 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:23,80 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Candidate to Leader 01:50:23,80 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 01:50:23,80 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 01:50:23,80 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: getLeaderAddress leaderId = member-1-shard-cars-testRaftCallbackDuringLeadershipDrop peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,80 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done false 01:50:23,81 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,81 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 01:50:23,81 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:23,81 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:23,81 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 01:50:23,81 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 01:50:23,81 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:23,118 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@62773f56: creating a new transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:50:23,118 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} 01:50:23,118 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:23,118 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:23,118 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1643686012] 01:50:23,118 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:23,118 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@47ba7d1b] 01:50:23,118 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} shard 1 01:50:23,118 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@47ba7d1b] 01:50:23,118 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:50:23,118 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:23,118 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@62773f56 readied transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:50:23,118 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6efb4a6c, attempting to connect 01:50:23,118 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2085976263], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@46488901 01:50:23,119 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:50:23,119 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} is not yet known 01:50:23,119 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, nanosAgo=9911, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:50:23,119 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:50:23,119 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:50:23,119 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:23,119 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@762195d1 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@6b183db6 01:50:23,119 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:50:23,119 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@6b183db6 01:50:23,119 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:50:23,119 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=OPEN} 01:50:23,119 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2085976263], modifications=1, protocol=SIMPLE} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=OPEN} 01:50:23,119 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2085976263], modifications=1, protocol=SIMPLE} 01:50:23,119 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SEALED} allocated sequence 0 01:50:23,119 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2085976263], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@46488901 01:50:23,119 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:50:23,119 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@21e8beff transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2085976263], coordinated=false}, enqueuedTicks=769453763992} 01:50:23,119 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@62773f56 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:23,120 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} 01:50:23,120 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@b37c2f9 01:50:23,120 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4935e70a superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@5e97c2b2, splicing queue 01:50:23,120 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4935e70a queue spliced 0 messages 01:50:23,120 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 653.3 μs 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: allocating new ready transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 initiating direct canCommit 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Validating transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 validated 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 initiating direct preCommit 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Preparing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 candidate ready 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1: canCommit - candidate rootPath: / 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1: canCommit - messages: [] 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 prepared 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 initiating direct commit 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Starting commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 174 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 01:50:23,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 submitted to persistence 01:50:23,121 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:50:23,121 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,121 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 01:50:23,121 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 1 01:50:23,121 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:23,121 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=174} 01:50:23,121 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:50:23,121 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 1 01:50:23,121 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:23,122 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:50:23,122 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done true 01:50:23,122 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 0, nextIndex: 1 01:50:23,122 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:23,122 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Setting last applied to 0 01:50:23,122 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 01:50:23,122 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data Simple{size=174} 01:50:23,122 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Resuming commit of transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:50:23,122 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@57529abd}, purged=MutableUnsignedLongSet{size=0}} 01:50:23,122 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 01:50:23,122 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:50:23,122 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1: commit - successfulFromPrevious: [] 01:50:23,122 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:50:23,122 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#209677336] with name member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 01:50:23,122 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:23,123 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=0 01:50:23,123 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2085976263], coordinated=false} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0}} 01:50:23,123 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2085976263], coordinated=false}, enqueuedTicks=769453763992, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0}} 01:50:23,123 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} directCommit completed 01:50:23,123 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 01:50:23,123 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#209677336] 01:50:23,123 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:50:23,123 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2085976263]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@3aec90d9 01:50:23,123 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} 01:50:23,123 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2085976263]} 01:50:23,123 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SEALED}: initiating purge 01:50:23,123 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=DONE} allocated sequence 1 01:50:23,123 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:23,123 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2085976263]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@f79bd77 01:50:23,123 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:50:23,123 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:23,123 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@21e8beff transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2085976263]}, enqueuedTicks=769458075673} 01:50:23,123 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#209677336] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,123 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 86.90 μs 01:50:23,123 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: purging transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:50:23,123 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 01:50:23,123 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 01:50:23,123 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:23,123 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 01:50:23,125 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:23,127 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 01:50:23,129 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 0 01:50:23,129 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:23,129 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 0 01:50:23,129 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 01:50:23,129 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data Simple{size=174} 01:50:23,129 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:50:23,130 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@46495444}, purged=MutableUnsignedLongSet{size=0}} 01:50:23,130 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 01:50:23,130 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:50:23,130 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-248169727] with name member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 01:50:23,130 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=0 01:50:23,130 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-248169727] 01:50:23,130 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:50:23,130 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-248169727] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,131 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 77.20 μs 01:50:23,131 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:50:23,132 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1613549502] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,132 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,132 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}]]] 01:50:23,133 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}]]] 01:50:23,133 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 1 01:50:23,133 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:23,133 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 01:50:23,133 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 01:50:23,133 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 2 01:50:23,133 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:23,135 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 01:50:23,135 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 1, nextIndex: 2 01:50:23,135 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:23,135 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Setting last applied to 1 01:50:23,135 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 01:50:23,135 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 01:50:23,135 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: replication of member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@4765b137 01:50:23,135 PM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: finished purging transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:50:23,135 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 01:50:23,135 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Purged transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:50:23,135 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:23,135 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 01:50:23,135 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 01:50:23,135 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:50:23,135 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2085976263]} with Envelope{sessionId=0, txSequence=1, message=TransactionPurgeResponse{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1}} 01:50:23,135 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1fbf0fd9 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@42666c79 01:50:23,135 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2085976263]}, enqueuedTicks=769458075673, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPurgeResponse{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1}} 01:50:23,135 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@764fa51 01:50:23,135 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=DONE}: purge completed 01:50:23,135 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 01:50:23,135 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@6b183db6 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=DONE} 01:50:23,135 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:50:23,135 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@762195d1 purged transaction RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} 01:50:23,135 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,136 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1710165023135, None)) 01:50:23,136 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165023134,0,0)) 01:50:23,136 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:23,136 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,136 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:23,136 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,136 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:23,136 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@42666c79 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@11f5dec7 01:50:23,136 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,136 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@a76f0b 01:50:23,136 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 01:50:23,137 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1613549502] 01:50:23,137 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:23,137 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,137 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,137 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,137 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#-1573392286] 01:50:23,137 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,137 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,137 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#-1573392286] 01:50:23,137 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,137 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,137 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#-1573392286] 01:50:23,137 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1376532396] 01:50:23,137 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 1 01:50:23,137 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 01:50:23,137 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:23,137 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 1 01:50:23,137 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 01:50:23,137 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 01:50:23,137 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 01:50:23,137 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Purged transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:50:23,137 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 01:50:23,137 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#-1573392286] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,137 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#312958685 created and ready for shard:member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,137 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-3:snapshot-read#1896671356] 01:50:23,137 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#312958685] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,137 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 01:50:23,137 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@62773f56: creating a new transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:50:23,137 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-3:snapshot-read#1896671356] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,138 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:50:23,138 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1643686012] 01:50:23,138 PM [pool-35-thread-1] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@62773f56 readied transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:50:23,138 PM [pool-35-thread-1] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=FLUSHED} allocated sequence 0 01:50:23,138 PM [pool-35-thread-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2085976263], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@413388e5 01:50:23,138 PM [pool-35-thread-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:23,138 PM [pool-35-thread-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@21e8beff transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2085976263], coordinated=false}, enqueuedTicks=769473481737} 01:50:23,138 PM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: allocating new ready transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 initiating direct canCommit 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Validating transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 validated 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 initiating direct preCommit 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Preparing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 candidate ready 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1: canCommit - candidate rootPath: / 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1: canCommit - messages: [] 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 prepared 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 initiating direct commit 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Starting commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 72 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}]]] 01:50:23,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 submitted to persistence 01:50:23,141 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:23,141 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:23,141 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:50:23,141 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:50:23,141 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 01:50:23,141 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 01:50:23,141 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 01:50:23,141 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=null 01:50:23,141 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 01:50:23,141 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: getLeaderAddress leaderId = member-1-shard-cars-testRaftCallbackDuringLeadershipDrop peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,141 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 01:50:23,141 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 01:50:23,141 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,142 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,142 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done false 01:50:23,142 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 01:50:23,142 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 01:50:23,142 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:23,142 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,142 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}]]] 01:50:23,142 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 01:50:23,142 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:23,142 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:23,142 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 01:50:23,142 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 01:50:23,239 PM [awaitility-thread] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542] 01:50:23,240 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542] 01:50:23,240 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:23,240 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Cluster state: [] 01:50:23,240 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:23,240 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Term 5 in "RequestVote [term=5, candidateId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1]" message is greater than leader's term 4 - switching to Follower 01:50:23,240 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=5, votedFor=null 01:50:23,240 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader) :- Switching from behavior Leader to Follower, election term: 5 01:50:23,240 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader)_slicer-id-680: Closing 01:50:23,240 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:50:23,240 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:50:23,240 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: removing frontend state for [member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop] 01:50:23,240 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 01:50:23,240 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=-1] 01:50:23,240 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Leader to Follower 01:50:23,240 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:23,240 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Leader to Follower 01:50:23,240 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:23,241 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@21e8beff superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@14f9c346, splicing queue 01:50:23,241 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:50:23,241 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@21e8beff queue spliced 1 messages 01:50:23,241 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:23,241 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: refreshing backend for shard 1 01:50:23,241 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:50:23,241 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:23,241 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:23,241 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1643686012] 01:50:23,241 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:23,241 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 01:50:23,487 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:23,487 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Received ElectionTimeout - switching to Candidate 01:50:23,487 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:50:23,487 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,487 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 6 01:50:23,487 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] to peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,487 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] to peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,487 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower) :- Switching from behavior Follower to Candidate, election term: 6 01:50:23,487 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 01:50:23,487 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:50:23,487 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:50:23,487 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Follower to Candidate 01:50:23,488 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Follower to Candidate 01:50:23,488 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized - notifying 1 OnShardInitialized callbacks 01:50:23,488 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:23,488 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 01:50:23,488 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:23,488 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 6 in "RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 4 - updating term 01:50:23,488 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=null 01:50:23,488 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 1, lastTerm: 4 01:50:23,488 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,488 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 01:50:23,489 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:23,489 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 01:50:23,489 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:23,489 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 6 in "RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 4 - updating term 01:50:23,489 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=null 01:50:23,489 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:23,489 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,489 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 01:50:23,489 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=true], my term is 6 01:50:23,489 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=true], current voteCount: 1 01:50:23,489 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): LastApplied index 1 is behind last index 2 - switching to PreLeader 01:50:23,489 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:50:23,489 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:23,489 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:23,489 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 6 01:50:23,489 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 01:50:23,489 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: became leader with frontend state for [member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop] 01:50:23,490 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:50:23,490 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Candidate to PreLeader 01:50:23,490 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:50:23,490 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 01:50:23,490 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Replicate message: identifier: null, logIndex: 3, payload: class org.opendaylight.controller.cluster.raft.persisted.NoopPayload, isSendImmediate: true 01:50:23,490 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized - notifying 1 OnShardInitialized callbacks 01:50:23,490 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:50:23,490 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:23,490 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:50:23,490 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Candidate to PreLeader 01:50:23,490 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized - notifying 1 OnShardInitialized callbacks 01:50:23,490 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:23,490 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): The followers log is empty and the senders prevLogIndex is 0 01:50:23,490 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:23,491 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): The followers log is empty and the senders prevLogIndex is 0 01:50:23,491 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:23,491 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done true 01:50:23,491 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:50:23,491 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 1, nextIndex: 2 01:50:23,491 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:50:23,491 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,491 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c]]] 01:50:23,491 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,492 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 01:50:23,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 01:50:23,492 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 01:50:23,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:50:23,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c]]] 01:50:23,492 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 01:50:23,492 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 01:50:23,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,492 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c]]] 01:50:23,492 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 2 01:50:23,492 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:50:23,492 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=72} 01:50:23,492 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 01:50:23,492 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c 01:50:23,492 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:50:23,492 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 01:50:23,492 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:23,492 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 01:50:23,492 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:50:23,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c]]] 01:50:23,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 4 01:50:23,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:23,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=174} 01:50:23,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:50:23,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 01:50:23,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 01:50:23,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=72} 01:50:23,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 01:50:23,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c 01:50:23,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:50:23,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 01:50:23,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 1 01:50:23,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:23,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 3 01:50:23,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 0 01:50:23,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 01:50:23,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data Simple{size=174} 01:50:23,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:50:23,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@47223f85}, purged=MutableUnsignedLongSet{size=0}} 01:50:23,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 01:50:23,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:50:23,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-85395690] with name member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 01:50:23,494 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-85395690] 01:50:23,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 1 01:50:23,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 01:50:23,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 01:50:23,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 01:50:23,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Purged transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:50:23,494 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:50:23,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 3, nextIndex: 4 01:50:23,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 4 does not match the current term 6 01:50:23,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 01:50:23,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Setting last applied to 2 01:50:23,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 01:50:23,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, size=72} 01:50:23,494 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 111.0 μs 01:50:23,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Resuming commit of transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:50:23,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 01:50:23,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:50:23,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1: commit - successfulFromPrevious: [] 01:50:23,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 01:50:23,494 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-85395690] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Setting last applied to 3 01:50:23,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 01:50:23,494 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} not found in inflight queue, checking pending queue 01:50:23,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c 01:50:23,494 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2085976263], coordinated=false} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} 01:50:23,495 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:50:23,495 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2085976263], coordinated=false}, enqueuedTicks=769473481737, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} 01:50:23,495 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=FLUSHED} directCommit completed 01:50:23,495 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Received ApplyState [identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}]] - lastApplied: 3, lastIndex: 3 01:50:23,495 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=FLUSHED}: initiating purge 01:50:23,495 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} allocated sequence 1 01:50:23,495 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2085976263]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@5e0eedc9 01:50:23,495 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:50:23,495 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:23,495 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:23,495 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:50:23,495 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,495 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:50:23,495 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=3, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c]]] 01:50:23,495 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 18.39 μs 01:50:23,495 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:50:23,495 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 6 01:50:23,495 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader)_slicer-id-683: Closing 01:50:23,495 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 01:50:23,495 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 01:50:23,495 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from PreLeader to Leader 01:50:23,495 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from PreLeader to Leader 01:50:23,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is ready - notifying 1 OnShardInitialized callbacks 01:50:23,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@47ba7d1b] 01:50:23,496 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 01:50:23,496 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@47ba7d1b] 01:50:23,496 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:50:23,496 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 01:50:23,496 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 01:50:23,496 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:50:23,496 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@3e1fe149, attempting to connect 01:50:23,496 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 01:50:23,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 3, nextIndex: 4 01:50:23,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 2 for term 6 01:50:23,496 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:50:23,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:50:23,496 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:50:23,496 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:23,496 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@6b183db6 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4b7e6bd 01:50:23,496 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} previous state DONE 01:50:23,496 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} replaying messages to old proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4b7e6bd 01:50:23,496 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:50:23,496 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=OPEN} 01:50:23,497 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Replaying queued request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2085976263]} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=OPEN} 01:50:23,497 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2085976263]} 01:50:23,497 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=OPEN}: initiating purge 01:50:23,497 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} allocated sequence 0 01:50:23,497 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2085976263]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@3dd10b14 01:50:23,497 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:50:23,497 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1418c9c2 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2085976263]}, enqueuedTicks=769829894351} 01:50:23,497 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@62773f56 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:23,497 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} 01:50:23,497 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@2fa157f2 01:50:23,497 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2ffbac75 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1ac79b35, splicing queue 01:50:23,497 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2ffbac75 queue spliced 0 messages 01:50:23,497 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 506.5 μs 01:50:23,497 PM [cluster-test-shard-dispatcher-27] [WARN] AbstractFrontendHistory - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 not tracked in MutableUnsignedLongSet{span=[0..0], size=1}, but not present in active transactions 01:50:23,498 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2085976263]} with Envelope{sessionId=1, txSequence=0, message=TransactionPurgeResponse{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} 01:50:23,498 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2085976263]}, enqueuedTicks=769829894351, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionPurgeResponse{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} 01:50:23,498 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE}: purge completed 01:50:23,498 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4b7e6bd purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} 01:50:23,498 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR}: purge completed 01:50:23,498 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@6b183db6 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} 01:50:23,498 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 3 01:50:23,498 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 01:50:23,498 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 2 01:50:23,498 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 01:50:23,498 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{size=72} 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=3, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c]]] 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 4 01:50:23,498 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] is present: existingEntryTerm: 4 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] is present: existingEntryTerm: 4 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] is present: existingEntryTerm: 4 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] is present: existingEntryTerm: 6 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 3, entries to be added from: 4 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 3 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:23,498 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 2 01:50:23,498 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{size=72} 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 01:50:23,498 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 3 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c 01:50:23,498 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 45.10 μs 01:50:23,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 01:50:23,498 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 3 01:50:23,498 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 01:50:23,498 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:50:23,498 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c 01:50:23,498 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 12.77 μs 01:50:23,499 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 01:50:23,542 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:50:23,542 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,542 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 01:50:23,542 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 01:50:23,543 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,543 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 01:50:23,543 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 01:50:23,543 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 01:50:23,543 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:50:23,543 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:50:23,543 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:50:23,544 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 01:50:23,544 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:50:23,544 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,544 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 01:50:23,544 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 01:50:23,544 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Initiating leader transfer 01:50:23,544 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#1044582918] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1044582918] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,544 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1613549502] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1613549502] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,544 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@427fc840 01:50:23,544 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@427fc840 01:50:23,544 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Attempting to transfer leadership 01:50:23,544 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:50:23,544 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:50:23,544 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=13] 01:50:23,544 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:23,545 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:23,545 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1418c9c2 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@45609459, splicing queue 01:50:23,545 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1418c9c2 queue spliced 0 messages 01:50:23,545 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:23,545 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: refreshing backend for shard 1 01:50:23,545 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:50:23,545 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:23,545 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:23,545 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1643686012] 01:50:23,545 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:23,545 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 01:50:23,547 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 2 for term 6 01:50:23,547 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:50:23,547 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=13] 01:50:23,547 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, matchIndex: 3, lastIndex: 3, isVoting: true 01:50:23,547 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Follower's log matches - sending ElectionTimeout 01:50:23,547 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:23,547 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:23,548 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:50:23,548 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:50:23,548 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Leader transfer complete 01:50:23,898 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:50:23,957 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 01:50:23,957 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-729723542]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:23,957 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received ShardNotInitializedTimeout message for shard cars 01:50:23,957 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Returning NoShardLeaderException for shard cars 01:50:23,957 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:23,957 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,957 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader)_slicer-id-684: Closing 01:50:23,958 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:23,958 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 01:50:23,958 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:23,958 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:23,958 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:50:23,959 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:50:23,959 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:50:23,959 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 01:50:23,959 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:50:23,959 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:50:23,959 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1643686012] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1643686012] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,960 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#2085976263] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#2085976263] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,960 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#2085976263] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#2085976263] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,960 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:23,960 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:23,963 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:23,963 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:23,963 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:23,963 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:23,965 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:23,981 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:50:23,981 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:23,981 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 01:50:23,981 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 01:50:23,982 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:50:23,987 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:23,987 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:24,978 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [5356127688951311061] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:25,8 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:25,8 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:25,8 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [6038874122084283199] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:25,10 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:25.011] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:25,19 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:25,19 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:25,21 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:25.022] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:25,46 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:50:25,51 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [7961126297630238234] 01:50:25,52 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:25,53 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:25,53 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:25,53 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:25,53 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:25,65 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:25,65 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:25,65 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:25,67 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:25,79 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [2656059109970994100] 01:50:25,79 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:25,88 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:25,88 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:25,88 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:25,88 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:25,107 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4150f92f aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} 01:50:25,107 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:50:25,107 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#725556861], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@77f98908 01:50:25,107 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} 01:50:25,107 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SEALED} allocated sequence 2 01:50:25,107 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#725556861], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x00000008013037f0@ec9a836 01:50:25,107 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:25,107 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#659728251], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:50:25,117 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:25,132 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [3768001722817699898] 01:50:25,132 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:25,147 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:25,147 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:25,147 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:25,147 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:25,151 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:25,153 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:25,155 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithSingleShard 01:50:25,165 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:25,166 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-790194198] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:25,167 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithSingleShard 01:50:25,169 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithSingleShard 01:50:25,169 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Creating local shard: member-1-shard-cars-testTransactionChainWithSingleShard 01:50:25,169 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: no initial generation override, starting from 0 01:50:25,169 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:50:25,169 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@42d3f690 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6294036f 01:50:25,170 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:25,170 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:50:25,171 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:25,176 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3834df1e 01:50:25,176 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionChainWithSingleShard, 0, 1710165025169, None)) 01:50:25,177 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165025168,0,0)) 01:50:25,177 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@11891b4b 01:50:25,177 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6294036f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2bbfc20 01:50:25,177 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} started 01:50:25,178 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:25,179 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:25,180 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:25,180 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionChainWithSingleShard 01:50:25,180 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:50:25,180 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:50:25,180 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: updated peer member-1-shard-cars-testTransactionChainWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard on shard member-1-shard-cars-testTransactionChainWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1215041890] 01:50:25,180 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithSingleShard 01:50:25,180 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionChainWithSingleShard, persistent : true 01:50:25,181 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1215041890] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:25,182 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionChainWithSingleShard with journal batch size 1 01:50:25,182 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-notifier#1431926936 created and ready for shard:member-1-shard-cars-testTransactionChainWithSingleShard 01:50:25,183 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-notifier#1431926936] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:25,184 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-member-1:snapshot-read#1706400253] 01:50:25,184 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-member-1:snapshot-read#1706400253] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:25,184 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#290859660] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:25,185 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithSingleShard 01:50:25,186 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: no initial generation override, starting from 0 01:50:25,186 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithSingleShard 01:50:25,186 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:50:25,186 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Creating local shard: member-2-shard-cars-testTransactionChainWithSingleShard 01:50:25,186 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@772a254c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5dbf4324 01:50:25,186 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@626cb929 01:50:25,186 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:25,186 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:50:25,186 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionChainWithSingleShard, 0, 1710165025186, None)) 01:50:25,187 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165025185,0,0)) 01:50:25,187 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@441e2473 01:50:25,187 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:25,187 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:25,187 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5dbf4324 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@e12ad5e 01:50:25,187 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Initializing shard [cars] 01:50:25,187 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionChainWithSingleShard is initialized 01:50:25,187 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} started 01:50:25,187 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:25,187 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithSingleShard is no longer the leader 01:50:25,187 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:25,187 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionChainWithSingleShard 01:50:25,187 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:50:25,188 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:50:25,188 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: updated peer member-1-shard-cars-testTransactionChainWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard on shard member-2-shard-cars-testTransactionChainWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard#-608741083] 01:50:25,188 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from null to Follower 01:50:25,188 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard 01:50:25,188 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionChainWithSingleShard, persistent : true 01:50:25,188 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard#-608741083] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:25,188 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/shard-member-2:snapshot-read#871967880] 01:50:25,188 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionChainWithSingleShard with journal batch size 1 01:50:25,188 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-notifier#-766855525] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:25,188 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/shard-member-2:snapshot-read#871967880] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:25,188 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-notifier#-766855525 created and ready for shard:member-2-shard-cars-testTransactionChainWithSingleShard 01:50:25,188 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from null to Follower 01:50:25,189 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:25,189 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:25,189 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:25,189 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionChainWithSingleShard is no longer the leader 01:50:25,189 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionChainWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:50:25,189 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Initializing shard [cars] 01:50:25,189 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionChainWithSingleShard is initialized 01:50:25,189 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithSingleShard , received role change from null to Follower 01:50:25,189 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard 01:50:25,189 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-2-shard-cars-testTransactionChainWithSingleShard from null to Follower 01:50:25,196 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:25,197 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:25,237 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-790194198] 01:50:25,237 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: findLocalShard : cars 01:50:25,238 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1215041890] 01:50:25,481 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:25,481 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 01:50:25,481 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionChainWithSingleShard] 01:50:25,481 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 01:50:25,481 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 1 01:50:25,481 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:25,481 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Closing 01:50:25,481 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:25,481 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithSingleShard is no longer the leader 01:50:25,484 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from Follower to Candidate 01:50:25,484 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from Follower to Candidate 01:50:25,715 PM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:25,791 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 01:50:25,791 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 01:50:25,791 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 2 01:50:26,72 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:26,72 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:26,72 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:26,72 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard 01:50:26,72 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:50:26,72 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:26,72 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:50:26,72 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:50:26,72 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionChainWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:50:26,78 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:26,78 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard 01:50:26,78 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:50:26,79 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:26,79 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:50:26,79 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:50:26,108 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 01:50:26,108 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 01:50:26,108 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 3 01:50:26,109 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionChainWithSingleShard 01:50:26,115 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:26,115 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Cluster state: [] 01:50:26,115 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:26,115 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:26,115 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=3, votedFor=null 01:50:26,115 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:26,115 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 01:50:26,115 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 01:50:26,116 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 01:50:26,116 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 01:50:26,116 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionChainWithSingleShard] 01:50:26,116 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:26,116 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 01:50:26,116 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:50:26,116 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: became leader with frontend state for [] 01:50:26,116 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from Candidate to Leader 01:50:26,116 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderPayloadVersion=13] 01:50:26,116 PM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:26,116 PM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:26,116 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from Candidate to Leader 01:50:26,116 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: All Shards are ready - data store testTransactionChainWithSingleShard is ready 01:50:26,116 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithSingleShard: Closing 01:50:26,116 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testTransactionChainWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:50:26,116 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard Received follower initial sync status for member-2-shard-cars-testTransactionChainWithSingleShard status sync done false 01:50:26,116 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderPayloadVersion=13] 01:50:26,116 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:26,116 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:26,116 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: All Shards are ready - data store testTransactionChainWithSingleShard is ready 01:50:26,117 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: -1, nextIndex: 0 01:50:26,117 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:26,142 PM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@70d781b8 01:50:26,142 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@70d781b8 changed state from IDLE to TX_OPEN 01:50:26,142 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} 01:50:26,142 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:26,142 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:26,142 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#290859660] 01:50:26,142 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} shard 1 01:50:26,142 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:50:26,143 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:50:26,143 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 01:50:26,143 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@644c0e 01:50:26,143 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@70d781b8 changed state from TX_OPEN to IDLE 01:50:26,143 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@70d781b8 readied transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:50:26,143 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:26,143 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 01:50:26,143 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 0 01:50:26,143 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@22f07118 01:50:26,143 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard, primaryVersion=13] 01:50:26,144 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard, primaryVersion=13] 01:50:26,144 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@2e6bf15f, attempting to connect 01:50:26,145 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} is not yet known 01:50:26,145 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, nanosAgo=10710, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:50:26,146 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:50:26,146 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1215041890], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:50:26,146 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1215041890], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:26,146 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@53ddbee3 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2000cbb8 01:50:26,146 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state SEALED 01:50:26,146 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} previous state OPEN 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1215041890], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13a179aa transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706]}, enqueuedTicks=772477720324} 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2000cbb8 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=1, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2218/0x00000008016a3c10@c7db774 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13a179aa transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=1, protocol=READY}, enqueuedTicks=772477933987} 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-687: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=1, protocol=READY}} 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-687: Message size: 390 bytes, total slices to send: 1 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-687: Message does not need to be sliced - sending original message 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2000cbb8 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 0 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1248/0x0000000801301d10@25d90b1c 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13a179aa transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=772478177371} 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@70d781b8 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1215041890], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@29fd612c 01:50:26,147 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@a8db415 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@6e799585, splicing queue 01:50:26,148 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@a8db415 queue spliced 0 messages 01:50:26,148 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1215041890], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.202 ms 01:50:26,148 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 01:50:26,148 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 01:50:26,148 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:50:26,148 PM [cluster-test-shard-dispatcher-17] [DEBUG] LeaderFrontendState - member-1-shard-cars-testTransactionChainWithSingleShard: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:50:26,149 PM [cluster-test-shard-dispatcher-17] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: allocating new open transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:50:26,149 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:50:26,149 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@7ddd46fd 01:50:26,149 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:50:26,149 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 ready 01:50:26,149 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE}}}]} 01:50:26,149 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: transitioned member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 to ready 01:50:26,149 PM [cluster-test-shard-dispatcher-17] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: allocating new open snapshot member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 01:50:26,149 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Reusing a chained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:50:26,149 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 snapshot MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE}}}] 01:50:26,150 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-686: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0}} 01:50:26,150 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-686: Message size: 331 bytes, total slices to send: 1 01:50:26,150 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-686: Message does not need to be sliced - sending original message 01:50:26,151 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 01:50:26,151 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0}} 01:50:26,151 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=1, protocol=READY}, enqueuedTicks=772477933987, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0}} 01:50:26,151 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0} 01:50:26,151 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0}} 01:50:26,151 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=772478177371, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0}} 01:50:26,151 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0} 01:50:26,151 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@70d781b8 changed state from IDLE to TX_OPEN 01:50:26,151 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:50:26,151 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 0 01:50:26,151 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@3bb96cd1 01:50:26,151 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13a179aa transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=772486694461} 01:50:26,156 PM [cluster-test-shard-dispatcher-17] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: allocating new open transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:50:26,156 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Reusing a chained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:50:26,156 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 snapshot MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}] 01:50:26,156 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-686: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0}} 01:50:26,156 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-686: Message size: 331 bytes, total slices to send: 1 01:50:26,156 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-686: Message does not need to be sliced - sending original message 01:50:26,157 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0}} 01:50:26,157 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=772486694461, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0}} 01:50:26,157 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0} 01:50:26,157 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 1 01:50:26,157 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 2 01:50:26,157 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=2} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@709da6ea 01:50:26,158 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13a179aa transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=2}, enqueuedTicks=772492778911} 01:50:26,158 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-687: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=2}} 01:50:26,158 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-687: Message size: 460 bytes, total slices to send: 1 01:50:26,158 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-687: Message does not need to be sliced - sending original message 01:50:26,158 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ExistsTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1240/0x00000008013035b8@10db0805 01:50:26,158 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13a179aa transmitting entry ConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]}, enqueuedTicks=772493054017} 01:50:26,159 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:26,160 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=2} with Envelope{sessionId=0, txSequence=4, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1}} 01:50:26,160 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=2}, enqueuedTicks=772492778911, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1}} 01:50:26,160 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=2} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1} 01:50:26,160 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request ExistsTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]} with Envelope{sessionId=0, txSequence=5, message=ExistsTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, exists=true}} 01:50:26,160 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]}, enqueuedTicks=772493054017, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=ExistsTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, exists=true}} 01:50:26,160 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteProxyTransaction - Exists request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] completed with ExistsTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, exists=true} 01:50:26,160 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 3 01:50:26,160 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@7a014bb 01:50:26,160 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13a179aa transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=772495628698} 01:50:26,162 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-686: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3}} 01:50:26,162 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-686: Message size: 369 bytes, total slices to send: 1 01:50:26,162 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-686: Message does not need to be sliced - sending original message 01:50:26,163 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3}} 01:50:26,163 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=772495628698, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3}} 01:50:26,163 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3} 01:50:26,163 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 4 01:50:26,163 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 01:50:26,163 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@79ac944d 01:50:26,163 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13a179aa transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=0, protocol=SIMPLE}, enqueuedTicks=772498369433} 01:50:26,163 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-687: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=0, protocol=SIMPLE}} 01:50:26,163 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-687: Message size: 286 bytes, total slices to send: 1 01:50:26,163 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-687: Message does not need to be sliced - sending original message 01:50:26,164 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 is already in state READY (READY) 01:50:26,164 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 initiating direct canCommit 01:50:26,164 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Validating transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:50:26,164 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 validated 01:50:26,164 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:26,164 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 initiating direct preCommit 01:50:26,164 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Preparing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:50:26,164 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 candidate ready 01:50:26,164 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1: canCommit - candidate rootPath: / 01:50:26,164 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1: canCommit - messages: [] 01:50:26,164 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1: preCommit - successfulFromPrevious: [] 01:50:26,164 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 prepared 01:50:26,164 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:26,164 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 initiating direct commit 01:50:26,164 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Starting commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:50:26,165 PM [cluster-test-shard-dispatcher-17] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 174 01:50:26,165 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=174}] 01:50:26,165 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=174}] 01:50:26,165 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:26,165 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:26,165 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 01:50:26,165 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}], SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=174}]]] 01:50:26,165 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 submitted to persistence 01:50:26,165 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=174}] 01:50:26,171 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}], SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=174}]]] 01:50:26,171 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 2 01:50:26,171 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:26,171 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 01:50:26,171 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 01:50:26,172 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log Simple{size=174} 01:50:26,172 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=174}] 01:50:26,172 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 2 01:50:26,172 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:26,172 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 01:50:26,172 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=174}] 01:50:26,172 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard Received follower initial sync status for member-2-shard-cars-testTransactionChainWithSingleShard status sync done true 01:50:26,172 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 1, nextIndex: 2 01:50:26,172 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 01:50:26,172 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 0 01:50:26,172 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from -1 to 0 01:50:26,172 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 01:50:26,172 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2220/0x00000008016a65d8@28815c37 01:50:26,172 PM [cluster-test-shard-dispatcher-17] [DEBUG] LeaderFrontendState - member-1-shard-cars-testTransactionChainWithSingleShard: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:50:26,172 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionChainWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@334a0306}, purged=MutableUnsignedLongSet{size=0}} 01:50:26,172 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:50:26,172 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 1 01:50:26,172 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 0 to 1 01:50:26,172 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 1 data Simple{size=174} 01:50:26,172 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Resuming commit of transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:50:26,173 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Committed transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:50:26,173 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1: commit - successfulFromPrevious: [] 01:50:26,173 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:50:26,173 PM [cluster-test-shard-dispatcher-17] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 01:50:26,173 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionChainWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1795889268] with name member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher 01:50:26,173 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1795889268] 01:50:26,173 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 0 for term 3 01:50:26,173 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 01:50:26,173 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:26,173 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=1 01:50:26,173 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 88.85 μs 01:50:26,173 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1795889268] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:26,173 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} 01:50:26,173 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706]}, enqueuedTicks=772477720324, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} 01:50:26,173 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0} 01:50:26,174 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=7, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1}} 01:50:26,174 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=0, protocol=SIMPLE}, enqueuedTicks=772498369433, sessionId=0, txSequence=7} with Envelope{sessionId=0, txSequence=7, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1}} 01:50:26,174 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} directCommit completed 01:50:26,174 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 1 01:50:26,174 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED}: initiating purge 01:50:26,174 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 2 01:50:26,174 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 01:50:26,174 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-436775706]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@395e2bd 01:50:26,174 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 0 01:50:26,174 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from -1 to 0 01:50:26,174 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1215041890], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:26,174 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 01:50:26,174 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13a179aa transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-436775706]}, enqueuedTicks=772509173831} 01:50:26,174 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionChainWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1f082940}, purged=MutableUnsignedLongSet{size=0}} 01:50:26,174 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:50:26,174 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 1 01:50:26,174 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 0 to 1 01:50:26,174 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 1 data Simple{size=174} 01:50:26,174 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithSingleShard: Applying foreign transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:50:26,174 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@542b2920 01:50:26,174 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13a179aa transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=2, protocol=READY}, enqueuedTicks=772509506657} 01:50:26,174 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-687: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=9, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=2, protocol=READY}} 01:50:26,175 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-687: Message size: 474 bytes, total slices to send: 1 01:50:26,175 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-687: Message does not need to be sliced - sending original message 01:50:26,175 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Committed transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:50:26,175 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@70d781b8 changed state from TX_OPEN to IDLE 01:50:26,175 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@70d781b8 readied transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:50:26,175 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=FLUSHED} allocated sequence 5 01:50:26,175 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionChainWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-389786434] with name member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher 01:50:26,175 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@7759cf0 01:50:26,175 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13a179aa transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=0, protocol=SIMPLE}, enqueuedTicks=772510398632} 01:50:26,175 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 0 for term 3 01:50:26,175 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-687: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=a, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=0, protocol=SIMPLE}} 01:50:26,175 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=1 01:50:26,175 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-687: Message size: 287 bytes, total slices to send: 1 01:50:26,175 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-687: Message does not need to be sliced - sending original message 01:50:26,175 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-389786434] 01:50:26,176 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 01:50:26,176 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-389786434] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:26,176 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 72.31 μs 01:50:26,178 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: purging transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:50:26,178 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=2, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 01:50:26,178 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 01:50:26,178 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:26,178 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:50:26,178 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 ready 01:50:26,178 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=MERGE, modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}], operation=MERGE}}}}}}}]} 01:50:26,178 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: transitioned member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 to ready 01:50:26,178 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 is already in state READY (READY) 01:50:26,178 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 initiating direct canCommit 01:50:26,178 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Validating transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:50:26,178 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 validated 01:50:26,178 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:26,178 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 initiating direct preCommit 01:50:26,178 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Preparing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:50:26,179 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 candidate ready 01:50:26,179 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1: canCommit - candidate rootPath: / 01:50:26,179 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1: canCommit - messages: [] 01:50:26,179 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1: preCommit - successfulFromPrevious: [] 01:50:26,179 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 prepared 01:50:26,179 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:26,179 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 initiating direct commit 01:50:26,179 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Starting commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:50:26,179 PM [cluster-test-shard-dispatcher-17] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 230 01:50:26,179 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=230}] 01:50:26,179 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=230}] 01:50:26,179 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:26,179 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 01:50:26,179 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 01:50:26,179 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=230}]]] 01:50:26,179 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 submitted to persistence 01:50:26,179 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 01:50:26,180 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=230}] 01:50:26,180 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=2, protocol=READY} with Envelope{sessionId=0, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4}} 01:50:26,180 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=2, protocol=READY}, enqueuedTicks=772509506657, sessionId=0, txSequence=9} with Envelope{sessionId=0, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4}} 01:50:26,180 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=2, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4} 01:50:26,180 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=230}]]] 01:50:26,180 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 2 01:50:26,180 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:50:26,180 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66} 01:50:26,180 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 01:50:26,180 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log Simple{size=230} 01:50:26,180 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=230}] 01:50:26,180 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 3 01:50:26,180 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:26,180 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 01:50:26,180 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=230}] 01:50:26,181 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 3, nextIndex: 4 01:50:26,181 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 01:50:26,181 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 2 01:50:26,181 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 1 to 2 01:50:26,181 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66} 01:50:26,181 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@478bff4e 01:50:26,181 PM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: finished purging transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:50:26,181 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Purged transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:50:26,181 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 3 01:50:26,181 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 2 to 3 01:50:26,181 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 3 data Simple{size=230} 01:50:26,181 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Resuming commit of transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:50:26,181 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Committed transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:50:26,181 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1: commit - successfulFromPrevious: [] 01:50:26,181 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 changing state from COMMIT_PENDING to COMMITTED 01:50:26,181 PM [cluster-test-shard-dispatcher-15] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=MERGE, modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}], operation=MERGE, modificationType=WRITE}}}}}}}]} 01:50:26,181 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 01:50:26,181 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 2 for term 3 01:50:26,181 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 77.53 μs 01:50:26,182 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:50:26,182 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=3 01:50:26,182 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 3 01:50:26,182 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 01:50:26,182 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 2 01:50:26,182 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 1 to 2 01:50:26,182 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66} 01:50:26,182 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Purged transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:50:26,182 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 3 01:50:26,182 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 2 to 3 01:50:26,182 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 3 data Simple{size=230} 01:50:26,183 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithSingleShard: Applying foreign transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:50:26,183 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-436775706]} with Envelope{sessionId=0, txSequence=8, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2}} 01:50:26,183 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-436775706]}, enqueuedTicks=772509173831, sessionId=0, txSequence=8} with Envelope{sessionId=0, txSequence=8, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2}} 01:50:26,183 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 01:50:26,183 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2000cbb8 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} 01:50:26,183 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5}} 01:50:26,183 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=0, protocol=SIMPLE}, enqueuedTicks=772510398632, sessionId=0, txSequence=10} with Envelope{sessionId=0, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5}} 01:50:26,183 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=FLUSHED} directCommit completed 01:50:26,183 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=FLUSHED}: initiating purge 01:50:26,183 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=DONE} allocated sequence 6 01:50:26,183 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-436775706]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@41f7de57 01:50:26,183 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1215041890], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:26,183 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13a179aa transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-436775706]}, enqueuedTicks=772518066637} 01:50:26,183 PM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:50:26,183 PM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 01:50:26,183 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Committed transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:50:26,183 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@53ddbee3 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} 01:50:26,183 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} allocated sequence 1 01:50:26,183 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@75dbf97c 01:50:26,183 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} 01:50:26,183 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 2 for term 3 01:50:26,183 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SEALED} allocated sequence 1 01:50:26,183 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SEALED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x00000008013037f0@4d1d30dc 01:50:26,183 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=3 01:50:26,183 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13a179aa transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=0, protocol=ABORT}, enqueuedTicks=772518397984} 01:50:26,183 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-687: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=c, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=0, protocol=ABORT}} 01:50:26,183 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-687: Message size: 287 bytes, total slices to send: 1 01:50:26,183 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-687: Message does not need to be sliced - sending original message 01:50:26,183 PM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:50:26,183 PM [main] [WARN] ClientBackedTransactionChain - Aborted unclosed transactions [member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0] java.lang.Throwable: at at org.opendaylight.controller.cluster.databroker.ClientBackedTransactionChain.close(ClientBackedTransactionChain.java:76) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.testTransactionChainWithSingleShard(DistributedDataStoreRemotingIntegrationTest.java:581) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) 01:50:26,184 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2000cbb8 invoking destroy 01:50:26,184 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13a179aa transmitting entry ConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706]}, enqueuedTicks=772518870141} 01:50:26,184 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@70d781b8 changed state from IDLE to CLOSED 01:50:26,184 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4859886b: creating a new snapshot member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 01:50:26,184 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=0, cookie=0} shard 1 01:50:26,184 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 01:50:26,184 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:26,184 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@669d8f40 01:50:26,184 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13a179aa transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=772519109265} 01:50:26,184 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 01:50:26,184 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 49.76 μs 01:50:26,185 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: purging transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:50:26,185 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=4, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 01:50:26,185 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 01:50:26,185 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:26,185 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: aborting transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 01:50:26,185 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=5, term=3, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 01:50:26,185 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=3, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 01:50:26,186 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 01:50:26,186 PM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: closing history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:50:26,186 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:50:26,186 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=6, term=3, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 01:50:26,186 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=3, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 01:50:26,186 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CloseLocalHistoryPayload, isSendImmediate: false 01:50:26,186 PM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: allocating new open snapshot member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1 01:50:26,186 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=4, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 01:50:26,186 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-686: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=e, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0}} 01:50:26,186 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-686: Message size: 369 bytes, total slices to send: 1 01:50:26,186 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-686: Message does not need to be sliced - sending original message 01:50:26,186 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=5, term=3, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 01:50:26,186 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=6, term=3, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 01:50:26,187 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=e, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0}} 01:50:26,187 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-436775706], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=772519109265, sessionId=0, txSequence=14} with Envelope{sessionId=0, txSequence=e, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0}} 01:50:26,187 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0} 01:50:26,188 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:50:26,188 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testTransactionChainWithSingleShard 01:50:26,188 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithSingleShard - waiting on 1 shards 01:50:26,188 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onShutDown 01:50:26,188 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Initiating leader transfer 01:50:26,188 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:50:26,188 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@45b45156 01:50:26,188 PM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:26,188 PM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:26,188 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:50:26,188 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@45b45156 01:50:26,188 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Attempting to transfer leadership 01:50:26,188 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 6, leaderSnapShotIndex: 2 01:50:26,188 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 01:50:26,188 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=3, prevLogTerm=3, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}], SimpleReplicatedLogEntry [index=5, term=3, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}], SimpleReplicatedLogEntry [index=6, term=3, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}]]] 01:50:26,189 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard] 01:50:26,189 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:50:26,189 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=3, prevLogTerm=3, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}], SimpleReplicatedLogEntry [index=5, term=3, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}], SimpleReplicatedLogEntry [index=6, term=3, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}]]] 01:50:26,189 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 3 01:50:26,189 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 01:50:26,189 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:26,189 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67} 01:50:26,189 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 01:50:26,189 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:26,189 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67} 01:50:26,189 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=3, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 01:50:26,189 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 01:50:26,189 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=3, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 01:50:26,189 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 4 01:50:26,189 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13a179aa superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@4f5580d, splicing queue 01:50:26,189 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=6, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:26,189 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1215041890], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:50:26,189 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@13a179aa queue spliced 3 messages 01:50:26,189 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1215041890], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1215041890], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:26,189 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: refreshing backend for shard 1 01:50:26,189 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1215041890], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:50:26,189 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:26,189 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:26,189 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#290859660] 01:50:26,189 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:26,189 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:50:26,189 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=4, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 01:50:26,189 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=5, term=3, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 01:50:26,189 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=6, term=3, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 6, nextIndex: 7 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 3 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 4 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 3 to 4 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67} 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@b63f15c 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: finished purging transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Purged transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 5 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 4 to 5 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 5 data AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67} 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2264/0x00000008016c74c8@6cba2f87 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Aborted transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 6 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 5 to 6 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 6 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2235/0x00000008016b63a0@3c275dba 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 5 for term 3 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionChainWithSingleShard, matchIndex: 6, lastIndex: 6, isVoting: true 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Leader transfer complete 01:50:26,190 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=6 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6}} not found in inflight queue, checking pending queue 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-436775706]} with Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6}} 01:50:26,191 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 6 01:50:26,191 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 3, lastIndex(): 6 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-436775706]}, enqueuedTicks=772518066637, sessionId=0, txSequence=11} with Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6}} 01:50:26,191 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 4 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=DONE}: purge completed 01:50:26,191 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 3 to 4 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2000cbb8 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=DONE} 01:50:26,191 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67} 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=c, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1}} not found in inflight queue, checking pending queue 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=c, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1}} 01:50:26,191 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Purged transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=0, protocol=ABORT}, enqueuedTicks=772518397984, sessionId=0, txSequence=12} with Envelope{sessionId=0, txSequence=c, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1}} 01:50:26,191 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 5 01:50:26,191 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 4 to 5 01:50:26,191 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 5 data AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67} 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=0, protocol=ABORT} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1} 01:50:26,191 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Aborted transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1} 01:50:26,191 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 6 01:50:26,191 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 5 to 6 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR}: initiating purge 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} allocated sequence 2 01:50:26,191 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 6 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 01:50:26,191 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:50:26,191 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 5 for term 3 01:50:26,191 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=6 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-436775706]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@4a41ab37 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SEALED}: initiating purge 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=DONE} allocated sequence 2 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-436775706]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@64daa742 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1215041890], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=d, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1}} not found in inflight queue, checking pending queue 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706]} with Envelope{sessionId=0, txSequence=d, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1}} 01:50:26,191 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706]}, enqueuedTicks=772518870141, sessionId=0, txSequence=13} with Envelope{sessionId=0, txSequence=d, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1}} 01:50:26,192 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2000cbb8 destroy completed with LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1} 01:50:26,192 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@70d781b8: removed destroyed proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2000cbb8 01:50:26,192 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1215041890], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:26,601 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: All shards shutdown - sending PoisonPill to self 01:50:26,601 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-testTransactionChainWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1215041890]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:26,602 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionChainWithSingleShard 01:50:26,602 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard (Leader)_slicer-id-689: Closing 01:50:26,602 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Aborting 0 pending queued transactions 01:50:26,602 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithSingleShard 01:50:26,602 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 01:50:26,602 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:50:26,603 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 01:50:26,603 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: shutting down 01:50:26,603 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 01:50:26,604 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:26,604 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:26,609 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:26,609 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:26,609 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:26,609 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:26,611 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:26,618 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithSingleShard: Closing 01:50:26,618 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionChainWithSingleShard 01:50:26,618 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Closing 01:50:26,618 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Aborting 0 pending queued transactions 01:50:26,618 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithSingleShard 01:50:26,619 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:26,619 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:27,619 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [7961126297630238234] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:27,638 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:27,638 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:50:27.640] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:27,643 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [2656059109970994100] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:27,648 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:27,648 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:27,651 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:27.652] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:27,667 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:27,672 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [7799545074001401271] 01:50:27,672 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:27,673 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:27,673 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:27,682 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:27,682 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:27,684 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:27,684 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:27,684 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:27,685 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:27,690 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [7688312763351624021] 01:50:27,690 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:27,692 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:27,692 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:27,701 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:27,701 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:27,706 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:27,713 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-5101504105859486150] 01:50:27,714 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:27,714 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:27,714 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:27,725 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:27,725 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:27,726 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:27,726 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:27,732 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:27,735 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 01:50:27,737 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 01:50:27,744 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:27,744 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:27,747 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:27.747] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:27,754 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:27,754 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:27,755 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:27.756] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:27,768 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:27,774 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-6803838169329745606] 01:50:27,774 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:27,775 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:27,775 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:27,775 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:27,775 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:27,787 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:27,787 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:27,787 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:27,789 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:27,793 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-9014647886296606055] 01:50:27,793 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:27,798 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:27,798 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:27,804 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:27,804 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:27,808 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:27,841 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [2299136909594317301] 01:50:27,841 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:27,848 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:27,848 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:27,849 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:27,849 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:27,853 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:27,854 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:27,856 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testForwardedReadyTransactionForwardedToLeader 01:50:27,874 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-2071639673] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:27,875 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:27,878 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:27,880 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 01:50:27,892 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-testForwardedReadyTransactionForwardedToLeader 01:50:27,892 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: no initial generation override, starting from 0 01:50:27,892 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Creating local shard: member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:27,892 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 01:50:27,892 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@ab38a91 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1db2d37f 01:50:27,892 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3c45342e 01:50:27,892 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:27,892 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:27,892 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, 0, 1710165027892, None)) 01:50:27,892 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165027891,0,0)) 01:50:27,892 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@76c46a91 01:50:27,892 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1db2d37f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3e80b717 01:50:27,892 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} started 01:50:27,893 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:27,895 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:27,897 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testForwardedReadyTransactionForwardedToLeader 01:50:27,898 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:27,898 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:27,915 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:27,915 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:27,915 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: updated peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284] 01:50:27,915 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Shard created : member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, persistent : true 01:50:27,915 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:27,916 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-308597532 created and ready for shard:member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:27,916 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-308597532] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:27,916 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-1:snapshot-read#-1479489282] 01:50:27,916 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Starting recovery for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with journal batch size 1 01:50:27,916 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-1:snapshot-read#-1479489282] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:27,918 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:27,918 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:27,919 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:27,919 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 01:50:27,919 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Initializing shard [cars] 01:50:27,919 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is initialized 01:50:27,919 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from null to Follower 01:50:27,919 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader 01:50:27,926 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from null to Follower 01:50:27,926 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1935375218] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:27,927 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 01:50:27,933 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testForwardedReadyTransactionForwardedToLeader 01:50:27,933 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: no initial generation override, starting from 0 01:50:27,933 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 01:50:27,933 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Creating local shard: member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:27,933 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@22fe8a6f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@653733cd 01:50:27,933 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:27,933 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:27,948 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:27,951 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:27,951 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4a8b6f28 01:50:27,951 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, 0, 1710165027933, None)) 01:50:27,952 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165027932,0,0)) 01:50:27,952 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@79a6f939 01:50:27,952 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@653733cd to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@143c620e 01:50:27,952 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} started 01:50:27,952 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:27,952 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:27,952 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:27,952 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:27,952 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: updated peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-63579329] 01:50:27,953 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, persistent : true 01:50:27,953 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with journal batch size 1 01:50:27,953 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-63579329] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:27,953 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#727989100 created and ready for shard:member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:27,953 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#727989100] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:27,954 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-2:snapshot-read#1698579145] 01:50:27,954 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-2:snapshot-read#1698579145] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:27,954 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:27,954 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:27,954 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:27,954 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 01:50:27,954 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Initializing shard [cars] 01:50:27,954 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader is initialized 01:50:27,954 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:27,954 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from null to Follower 01:50:27,954 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader 01:50:27,954 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader from null to Follower 01:50:28,2 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-2071639673] 01:50:28,3 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 01:50:28,3 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284] 01:50:28,226 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:28,226 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Received ElectionTimeout - switching to Candidate 01:50:28,226 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] 01:50:28,226 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:28,226 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 1 01:50:28,226 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:28,226 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Closing 01:50:28,226 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:28,226 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 01:50:28,226 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from Follower to Candidate 01:50:28,226 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from Follower to Candidate 01:50:28,506 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:50:28,506 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:28,506 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 2 01:50:28,518 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:28,756 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:50:28,756 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:28,756 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 3 01:50:28,786 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:28,787 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:28,787 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:28,787 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:28,787 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:28,787 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:28,787 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:28,787 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:28,787 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Peer address for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:28,792 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:28,792 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:28,792 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:28,792 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:28,792 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:28,792 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:28,865 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:29,16 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:50:29,16 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:29,16 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 4 01:50:29,16 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:29,17 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:29,18 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Cluster state: [] 01:50:29,18 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:29,18 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:29,18 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=null 01:50:29,18 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:29,18 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:29,18 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:29,19 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:29,19 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:29,19 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] 01:50:29,19 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:29,19 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:29,19 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:50:29,19 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: became leader with frontend state for [] 01:50:29,19 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from Candidate to Leader 01:50:29,19 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderPayloadVersion=13] 01:50:29,19 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:29,19 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:29,19 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from Candidate to Leader 01:50:29,19 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All Shards are ready - data store testForwardedReadyTransactionForwardedToLeader is ready 01:50:29,20 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Closing 01:50:29,20 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:29,20 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader status sync done false 01:50:29,20 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderPayloadVersion=13] 01:50:29,20 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:29,20 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:29,20 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All Shards are ready - data store testForwardedReadyTransactionForwardedToLeader is ready 01:50:29,21 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: -1, nextIndex: 0 01:50:29,21 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:29,108 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1935375218] 01:50:29,108 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 01:50:29,109 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-63579329] 01:50:29,109 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:29,109 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1935375218] 01:50:29,109 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 01:50:29,109 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-63579329] 01:50:29,120 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: handleForwardedReadyTransaction for member-1-ShardTransactionTest-fe-0-txn-275-0 01:50:29,120 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:29,121 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader)] 01:50:29,122 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, disabling transaction tracking 01:50:29,123 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:50:29,123 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:50:29,123 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:29,123 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:29,123 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:29,123 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 01:50:29,123 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying 3 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-275-0 01:50:29,123 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-275-0 of 3 operations, client version 13 01:50:29,123 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-275-0 01:50:29,123 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-275-0 ready 01:50:29,123 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-275-0 01:50:29,123 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 validated 01:50:29,123 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:29,123 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-275-0: success 01:50:29,123 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-275-0 01:50:29,123 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-275-0 01:50:29,123 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 candidate ready 01:50:29,124 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-275-0: canCommit - candidate rootPath: / 01:50:29,124 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-275-0: canCommit - messages: [] 01:50:29,124 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-275-0: preCommit - successfulFromPrevious: [] 01:50:29,124 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 prepared 01:50:29,124 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:29,124 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-275-0 01:50:29,124 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-275-0 01:50:29,124 PM [cluster-test-shard-dispatcher-14] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 198 01:50:29,124 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}] 01:50:29,124 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}] 01:50:29,124 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-275-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:29,124 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:29,124 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:29,124 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 submitted to persistence 01:50:29,125 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 01:50:29,125 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:50:29,125 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:29,125 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:50:29,125 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:50:29,125 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 1 01:50:29,125 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:29,126 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:50:29,126 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}] 01:50:29,128 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:50:29,128 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader status sync done true 01:50:29,128 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 0, nextIndex: 1 01:50:29,128 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:29,128 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 0 01:50:29,128 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from -1 to 0 01:50:29,128 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:50:29,128 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 01:50:29,129 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 01:50:29,129 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, nanosAgo=6074910} is already disabled 01:50:29,129 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:29,129 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:29,129 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}]]] 01:50:29,129 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 01:50:29,129 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}]]] 01:50:29,129 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:50:29,129 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:29,129 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log Simple{size=198} 01:50:29,129 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}] 01:50:29,130 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 2 01:50:29,130 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 0 01:50:29,130 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:29,130 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:50:29,130 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 0 01:50:29,130 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from -1 to 0 01:50:29,130 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:50:29,130 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 01:50:29,130 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 01:50:29,130 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}] 01:50:29,130 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 01:50:29,130 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 1, nextIndex: 2 01:50:29,130 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:29,130 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 1 01:50:29,130 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 0 to 1 01:50:29,130 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=198} 01:50:29,130 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-275-0 01:50:29,130 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-275-0: commit - successfulFromPrevious: [] 01:50:29,130 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 changing state from COMMIT_PENDING to COMMITTED 01:50:29,130 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/system/testProbe-765#-78861002] 01:50:29,131 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-1344548049] with name member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher 01:50:29,131 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 01:50:29,131 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:29,131 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 01:50:29,131 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-1344548049] 01:50:29,131 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 01:50:29,131 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 61.08 μs 01:50:29,131 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-1344548049] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,131 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3a674e07: creating a new snapshot member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 01:50:29,132 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} 01:50:29,132 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:29,132 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:29,132 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-2071639673] 01:50:29,132 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:29,132 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, history=0, cookie=0} shard 1 01:50:29,132 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1 for transaction member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 01:50:29,132 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:29,132 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@779de616] 01:50:29,132 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-630665924], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@70dea80e 01:50:29,132 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:50:29,132 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@779de616] 01:50:29,132 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 1 01:50:29,132 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:29,132 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 1 01:50:29,132 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 0 to 1 01:50:29,132 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=198} 01:50:29,132 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-275-0 01:50:29,133 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#65824908] with name member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher 01:50:29,133 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 01:50:29,133 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 01:50:29,133 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6b981bc7, attempting to connect 01:50:29,133 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: client ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} is not yet known 01:50:29,133 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, nanosAgo=5730, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 01:50:29,133 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:50:29,133 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:50:29,133 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:29,133 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@53b028c9 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@79299579 01:50:29,133 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:50:29,133 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@79299579 01:50:29,133 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1 for transaction member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1 01:50:29,133 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 01:50:29,133 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-630665924], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 01:50:29,133 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3a674e07 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:29,133 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} 01:50:29,133 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@67d185b3 01:50:29,133 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@571a4c13 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@622d99d4, splicing queue 01:50:29,133 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@571a4c13 queue spliced 0 messages 01:50:29,133 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 317.8 μs 01:50:29,134 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0} 01:50:29,134 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: handleForwardedReadyTransaction for member-1-ShardTransactionTest-fe-0-txn-276-0 01:50:29,134 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:29,134 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader)] 01:50:29,135 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#65824908] 01:50:29,135 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 01:50:29,135 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 119.1 μs 01:50:29,135 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#65824908] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,136 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-276-0 01:50:29,136 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-276-0 of 1 operations, client version 13 01:50:29,136 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-276-0 01:50:29,137 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-276-0 ready 01:50:29,137 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0 canCommit 01:50:29,137 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284)] resolved 01:50:29,137 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: successfully resolved all cohort actors 01:50:29,137 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0 finishCanCommit 01:50:29,137 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: sending CanCommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-276-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284)] 01:50:29,137 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284)] 01:50:29,137 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Can committing transaction member-1-ShardTransactionTest-fe-0-txn-276-0 01:50:29,137 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-276-0 01:50:29,137 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 validated 01:50:29,137 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:29,137 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-276-0: success 01:50:29,137 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:50:29,137 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: canCommit returning result: true 01:50:29,137 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0 commit 01:50:29,137 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0 finish commit 01:50:29,137 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: Sending CommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-276-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284)] 01:50:29,138 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284)] 01:50:29,138 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-276-0 01:50:29,138 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-276-0 01:50:29,138 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 candidate ready 01:50:29,138 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-276-0: canCommit - candidate rootPath: / 01:50:29,138 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-276-0: canCommit - messages: [] 01:50:29,138 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-276-0: preCommit - successfulFromPrevious: [] 01:50:29,138 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 prepared 01:50:29,138 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:29,138 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-276-0 01:50:29,138 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-276-0 01:50:29,138 PM [cluster-test-shard-dispatcher-14] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 209 01:50:29,138 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}] 01:50:29,138 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}] 01:50:29,138 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-276-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:29,138 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:50:29,138 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:29,138 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}]]] 01:50:29,138 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 submitted to persistence 01:50:29,138 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}] 01:50:29,140 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}]]] 01:50:29,140 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:50:29,140 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:50:29,140 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log Simple{size=209} 01:50:29,140 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}] 01:50:29,140 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 2 01:50:29,140 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:29,140 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}] 01:50:29,140 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 2, nextIndex: 3 01:50:29,140 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:50:29,140 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 2 01:50:29,140 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 1 to 2 01:50:29,140 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=209} 01:50:29,140 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-276-0 01:50:29,140 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-276-0: commit - successfulFromPrevious: [] 01:50:29,140 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 changing state from COMMIT_PENDING to COMMITTED 01:50:29,140 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testForwardedReadyTransactionForwardedToLeader_member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader%23-648142284$a] 01:50:29,141 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 01:50:29,141 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:29,141 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 01:50:29,141 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 01:50:29,141 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: commit succeeded 01:50:29,141 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 58.16 μs 01:50:29,141 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3a674e07: creating a new snapshot member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0 01:50:29,141 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-1 for transaction member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0 01:50:29,141 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 01:50:29,141 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 2 01:50:29,141 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:50:29,141 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 2 01:50:29,141 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 1 to 2 01:50:29,141 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=209} 01:50:29,141 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-276-0 01:50:29,142 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 01:50:29,142 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Issuing gracefulStop to shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:29,142 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 01:50:29,142 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader - waiting on 1 shards 01:50:29,142 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onShutDown 01:50:29,142 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Initiating leader transfer 01:50:29,142 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1084aa29 01:50:29,142 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 01:50:29,142 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1084aa29 01:50:29,142 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Attempting to transfer leadership 01:50:29,142 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 53.37 μs 01:50:29,142 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:29,142 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, matchIndex: 2, lastIndex: 2, isVoting: true 01:50:29,142 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 01:50:29,142 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Follower's log matches - sending ElectionTimeout 01:50:29,142 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:29,142 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Leader transfer complete 01:50:29,142 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:29,142 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:29,142 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@45670808 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@74f8cfca, splicing queue 01:50:29,142 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@45670808 queue spliced 0 messages 01:50:29,143 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:29,143 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: refreshing backend for shard 1 01:50:29,143 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:50:29,143 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:29,143 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:29,143 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-2071639673] 01:50:29,143 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:29,143 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader] 01:50:29,143 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 01:50:29,143 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 01:50:29,143 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:29,143 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:29,556 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 01:50:29,556 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received ShardNotInitializedTimeout message for shard cars 01:50:29,556 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Returning NoShardLeaderException for shard cars 01:50:29,556 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:29,556 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:29,556 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:29,556 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader)_slicer-id-694: Closing 01:50:29,557 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:29,557 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:29,557 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Aborting 0 pending queued transactions 01:50:29,557 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:29,558 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 01:50:29,558 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 01:50:29,558 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 01:50:29,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: shutting down 01:50:29,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 01:50:29,559 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 01:50:29,559 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-2071639673] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-2071639673] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,559 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-2071639673] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-2071639673] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,559 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#-630665924] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-630665924] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,559 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-2071639673] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-2071639673] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,559 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#-630665924] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-630665924] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,559 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:29,559 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:29,565 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:29,565 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:29,565 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:29,565 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:29,566 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:29,577 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 01:50:29,577 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:29,577 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Closing 01:50:29,577 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Aborting 0 pending queued transactions 01:50:29,577 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 01:50:29,578 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:29,578 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:30,574 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-6803838169329745606] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:30,594 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-9014647886296606055] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:30,600 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:30,600 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:30,602 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:30.604] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:30,609 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:30,609 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:30,611 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:30.611] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:30,623 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:50:30,628 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4449430654510267557] 01:50:30,628 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:30,630 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:30,630 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:30,636 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:30,636 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:30,644 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:30,646 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:30,646 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:30,646 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:30,658 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-2399595631317018924] 01:50:30,658 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:30,658 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:30,658 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:30,658 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:30,664 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:30,668 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:50:30,676 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-1168583544366362752] 01:50:30,676 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:30,677 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:30,677 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:30,679 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:30,679 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:30,688 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:30,689 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:30,690 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:30,692 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:30,693 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithSingleShard 01:50:30,701 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-672051887] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:30,706 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 01:50:30,708 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: no initial generation override, starting from 0 01:50:30,708 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:50:30,708 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithSingleShard 01:50:30,708 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@51b3c5a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@42f1550f 01:50:30,708 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Creating local shard: member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:30,708 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@30eb18e2 01:50:30,708 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, 0, 1710165030708, None)) 01:50:30,709 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:30,709 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165030707,0,0)) 01:50:30,709 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:30,709 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@c73fd0 01:50:30,709 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@42f1550f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@28e6a64 01:50:30,709 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} started 01:50:30,709 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:30,709 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:30,709 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:30,709 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:30,709 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:30,709 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: updated peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard on shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#466558309] 01:50:30,710 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Shard created : member-1-shard-cars-testChainedTransactionFailureWithSingleShard, persistent : true 01:50:30,710 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#466558309] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:30,710 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-2064932948 created and ready for shard:member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:30,710 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-2064932948] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:30,710 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-1:snapshot-read#-154083727] 01:50:30,710 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:30,711 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Starting recovery for member-1-shard-cars-testChainedTransactionFailureWithSingleShard with journal batch size 1 01:50:30,711 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-1:snapshot-read#-154083727] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:30,712 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithSingleShard 01:50:30,713 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:30,713 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:30,713 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Initializing shard [cars] 01:50:30,713 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is initialized 01:50:30,713 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:30,713 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 01:50:30,713 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from null to Follower 01:50:30,713 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard 01:50:30,713 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from null to Follower 01:50:30,713 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:30,717 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 01:50:30,717 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#743377262] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:30,718 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: no initial generation override, starting from 0 01:50:30,718 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:50:30,718 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4587aca8 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@279a6588 01:50:30,718 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6298ef1f 01:50:30,718 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, 0, 1710165030718, None)) 01:50:30,719 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithSingleShard 01:50:30,719 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165030717,0,0)) 01:50:30,719 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Creating local shard: member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:30,719 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@279a6588 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@e97b21c 01:50:30,719 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:30,719 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:30,719 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@13677ed1 01:50:30,719 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} started 01:50:30,720 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:30,720 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:30,720 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:30,720 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:30,720 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: updated peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard on shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard#291720459] 01:50:30,720 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard#291720459] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:30,720 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Shard created : member-2-shard-cars-testChainedTransactionFailureWithSingleShard, persistent : true 01:50:30,721 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-1447757464 created and ready for shard:member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:30,721 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-1447757464] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:30,721 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-2:snapshot-read#1217212245] 01:50:30,721 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Starting recovery for member-2-shard-cars-testChainedTransactionFailureWithSingleShard with journal batch size 1 01:50:30,721 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-2:snapshot-read#1217212245] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:30,721 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:30,721 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:30,721 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Initializing shard [cars] 01:50:30,721 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard is initialized 01:50:30,721 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:30,721 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 01:50:30,721 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Peer address for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:30,722 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from null to Follower 01:50:30,722 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard 01:50:30,722 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-2-shard-cars-testChainedTransactionFailureWithSingleShard from null to Follower 01:50:30,728 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:30,769 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-672051887] 01:50:30,770 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: findLocalShard : cars 01:50:30,770 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#466558309] 01:50:31,10 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:31,10 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 01:50:31,10 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testChainedTransactionFailureWithSingleShard] 01:50:31,10 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:31,10 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 1 01:50:31,11 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:31,11 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Closing 01:50:31,11 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:31,11 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 01:50:31,11 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from Follower to Candidate 01:50:31,11 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from Follower to Candidate 01:50:31,273 PM [cluster-test-shard-dispatcher-18] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:31,300 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 01:50:31,300 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:31,300 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 2 01:50:31,570 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 01:50:31,571 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:31,571 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 3 01:50:31,651 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:31,651 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:31,652 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:31,652 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:31,652 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:31,652 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:31,652 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:31,652 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:31,652 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:31,656 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:31,656 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:31,656 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:31,656 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:31,657 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:31,657 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:31,684 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@79299579 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-1, state=OPEN} 01:50:31,684 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-630665924]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@6b465961 01:50:31,684 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 01:50:31,684 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:50:31,684 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@53b028c9 aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} 01:50:31,684 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:50:31,684 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-630665924], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@67412bd8 01:50:31,684 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} 01:50:31,684 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@79299579 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 01:50:31,684 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-630665924]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@5ccb5ff1 01:50:31,684 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 01:50:31,684 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-648142284], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:50:31,684 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4ab69830 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} 01:50:31,684 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 1 01:50:31,684 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-436775706], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@1f00c446 01:50:31,684 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1215041890], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:31,684 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0} 01:50:31,692 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:31,880 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 01:50:31,880 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:31,880 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 4 01:50:31,880 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:31,881 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:31,881 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Cluster state: [] 01:50:31,881 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:31,881 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:31,881 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=null 01:50:31,881 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:31,881 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:31,881 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:31,882 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:31,882 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:31,882 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testChainedTransactionFailureWithSingleShard] 01:50:31,883 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:31,883 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:31,883 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:50:31,883 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: became leader with frontend state for [] 01:50:31,883 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from Candidate to Leader 01:50:31,883 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderPayloadVersion=13] 01:50:31,883 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:31,883 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:31,883 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from Candidate to Leader 01:50:31,883 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All Shards are ready - data store testChainedTransactionFailureWithSingleShard is ready 01:50:31,883 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Closing 01:50:31,883 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testChainedTransactionFailureWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:31,884 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithSingleShard status sync done false 01:50:31,884 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderPayloadVersion=13] 01:50:31,884 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:31,884 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:31,884 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All Shards are ready - data store testChainedTransactionFailureWithSingleShard is ready 01:50:31,884 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: -1, nextIndex: 0 01:50:31,884 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:31,886 PM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3f962e04 01:50:31,886 PM [main] [DEBUG] AbstractDOMBroker - Transaction chain 0 created with listener Mock for DOMTransactionChainListener, hashCode: 1764706747, backing store chains {CONFIGURATION=org.opendaylight.controller.cluster.databroker.ClientBackedTransactionChain@35a4bd8d} 01:50:31,886 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3f962e04 changed state from IDLE to TX_OPEN 01:50:31,886 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} 01:50:31,886 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:31,886 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:31,886 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#743377262] 01:50:31,886 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0} shard 1 01:50:31,886 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:50:31,886 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 01:50:31,886 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 01:50:31,886 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1009384149], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2216/0x00000008016a2bf8@9ade45e 01:50:31,886 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3f962e04 changed state from TX_OPEN to IDLE 01:50:31,886 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3f962e04 readied transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 01:50:31,886 PM [main] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 is submitted for execution. 01:50:31,886 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 01:50:31,886 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1009384149], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@d7c4b37 01:50:31,887 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:31,887 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard, primaryVersion=13] 01:50:31,887 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard, primaryVersion=13] 01:50:31,888 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@4076ee, attempting to connect 01:50:31,889 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} is not yet known 01:50:31,889 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, nanosAgo=8160, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:50:31,890 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:50:31,891 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#466558309], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:50:31,891 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#466558309], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:31,891 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4b56a682 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@610852d8 01:50:31,891 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:50:31,891 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#466558309], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:50:31,891 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@77afc97f transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1009384149]}, enqueuedTicks=778221492945} 01:50:31,891 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@610852d8 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1009384149], modifications=1, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1009384149], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2218/0x00000008016a3c10@59311f0a 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@77afc97f transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1009384149], modifications=1, protocol=READY}, enqueuedTicks=778221634707} 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-697: slice: identifier: member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1009384149], modifications=1, protocol=READY}} 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-697: Message size: 487 bytes, total slices to send: 1 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-697: Message does not need to be sliced - sending original message 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1009384149], modifications=0, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 1 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1009384149], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2258/0x00000008016bac68@22765cd6 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@77afc97f transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1009384149], modifications=0, protocol=SIMPLE}, enqueuedTicks=778221704339} 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-697: slice: identifier: member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1009384149], modifications=0, protocol=SIMPLE}} 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-697: Message size: 295 bytes, total slices to send: 1 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-697: Message does not need to be sliced - sending original message 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3f962e04 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#466558309], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@1263be8a 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@781918 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1f2f6d6, splicing queue 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@781918 queue spliced 0 messages 01:50:31,892 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#466558309], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.103 ms 01:50:31,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 01:50:31,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 01:50:31,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:50:31,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] LeaderFrontendState - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 01:50:31,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: allocating new open transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 01:50:31,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 01:50:31,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@7f27775a 01:50:31,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendTransaction - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1009384149], modifications=1, protocol=READY} failed to process org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:31,895 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: request Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1009384149], modifications=1, protocol=READY}} failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1009384149], modifications=1, protocol=READY} failed to process at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:147) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ... 33 more 01:50:31,895 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: request Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1009384149], modifications=0, protocol=SIMPLE}} failed org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:31,895 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 01:50:31,908 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1009384149], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1009384149], modifications=1, protocol=READY} failed to process}} 01:50:31,908 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1009384149], modifications=1, protocol=READY}, enqueuedTicks=778221634707, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1009384149], modifications=1, protocol=READY} failed to process}} 01:50:31,908 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1009384149], modifications=1, protocol=READY} completed with TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1009384149], modifications=1, protocol=READY} failed to process} 01:50:31,908 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1009384149], modifications=1, protocol=READY} failed to process at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:147) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ... 33 more 01:50:31,908 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1009384149], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=2, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 01:50:31,908 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1009384149], modifications=0, protocol=SIMPLE}, enqueuedTicks=778221704339, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 01:50:31,908 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1009384149], modifications=0, protocol=SIMPLE} completed with TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0} 01:50:31,908 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 failed org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:31,908 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during phase CAN_COMMIT, starting Abort org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:31,908 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DOMBrokerTransactionChain - Transaction chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@26554813 failed. TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0]]} at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1119) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction.lambda$directCommit$4(AbstractProxyTransaction.java:517) at org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction.lambda$handleReplayedModifyTransactionRequest$16(RemoteProxyTransaction.java:509) at org.opendaylight.controller.cluster.access.client.ConnectionEntry.complete(ConnectionEntry.java:47) at org.opendaylight.controller.cluster.access.client.AbstractClientConnection.receiveResponse(AbstractClientConnection.java:494) at org.opendaylight.controller.cluster.access.client.AbstractReceivingClientConnection.receiveResponse(AbstractReceivingClientConnection.java:68) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onResponse(ClientActorBehavior.java:184) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onRequestFailure(ClientActorBehavior.java:196) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.internalOnRequestFailure(ClientActorBehavior.java:248) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:155) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:48) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.onReceiveCommand(AbstractClientActor.java:74) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) ... 17 more 01:50:31,909 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 aborted successfully 01:50:31,909 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} directCommit completed 01:50:31,909 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: initiating purge 01:50:31,909 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:50:31,909 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1009384149]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@5288df65 01:50:31,909 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} 01:50:31,909 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED}: initiating purge 01:50:31,909 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 2 01:50:31,909 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1009384149]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@1d93fba2 01:50:31,909 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@77afc97f transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1009384149]}, enqueuedTicks=778244050116} 01:50:31,909 PM [main] [DEBUG] DOMBrokerTransactionChain - Chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@26554813 is no longer running 01:50:31,909 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:50:31,909 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:31,909 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithSingleShard - waiting on 1 shards 01:50:31,910 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onShutDown 01:50:31,910 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Initiating leader transfer 01:50:31,910 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@75ba269a 01:50:31,910 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 01:50:31,910 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@75ba269a 01:50:31,910 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Attempting to transfer leadership 01:50:31,910 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:31,910 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:31,910 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:50:31,910 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}]]] 01:50:31,910 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:31,910 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:31,912 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: not currently active leader, rejecting request Envelope{sessionId=0, txSequence=3, message=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1009384149]}}. isLeader: true, isLeaderActive: false,isLeadershipTransferInProgress: true, paused: true 01:50:31,912 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: request Envelope{sessionId=0, txSequence=3, message=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1009384149]}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#466558309] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:31,913 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard] 01:50:31,913 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:50:31,913 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}]]] 01:50:31,913 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Number of entries to be appended = 1 01:50:31,913 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:31,913 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:31,913 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74} 01:50:31,913 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 01:50:31,913 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:31,913 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@77afc97f superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@29d7c055, splicing queue 01:50:31,913 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Log size is now 1 01:50:31,913 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#466558309], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:50:31,913 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:31,913 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@77afc97f queue spliced 2 messages 01:50:31,913 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#466558309], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#466558309], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:31,913 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: refreshing backend for shard 1 01:50:31,913 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#466558309], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:50:31,913 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:31,913 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:31,913 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#743377262] 01:50:31,913 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 01:50:31,913 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:31,913 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 01:50:31,913 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithSingleShard status sync done true 01:50:31,915 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: 0, nextIndex: 1 01:50:31,915 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:31,915 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Setting last applied to 0 01:50:31,915 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from -1 to 0 01:50:31,915 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74} 01:50:31,915 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2220/0x00000008016a65d8@624fc46a 01:50:31,915 PM [cluster-test-shard-dispatcher-24] [DEBUG] LeaderFrontendState - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 01:50:31,915 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendMetadata - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@11860c0a}, purged=MutableUnsignedLongSet{size=0}} 01:50:31,915 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 01:50:31,915 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:31,915 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testChainedTransactionFailureWithSingleShard, matchIndex: 0, lastIndex: 0, isVoting: true 01:50:31,915 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 01:50:31,915 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:31,915 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Leader transfer complete 01:50:31,915 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=0 01:50:31,916 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Commit index set to 0 01:50:31,916 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:31,916 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Setting last applied to 0 01:50:31,916 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from -1 to 0 01:50:31,916 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74} 01:50:31,916 PM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendMetadata - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@7ba883d2}, purged=MutableUnsignedLongSet{size=0}} 01:50:31,916 PM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 01:50:31,916 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=0 01:50:31,916 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} not found in inflight queue, checking pending queue 01:50:31,916 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1009384149]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} 01:50:31,916 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1009384149]}, enqueuedTicks=778221492945, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} 01:50:31,916 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0} 01:50:32,122 PM [cluster-test-shard-dispatcher-24] [WARN] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 204, lastApplied : 0, commitIndex : 0 01:50:32,320 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All shards shutdown - sending PoisonPill to self 01:50:32,320 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#466558309]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:32,320 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:32,320 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader)_slicer-id-699: Closing 01:50:32,321 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Aborting 0 pending queued transactions 01:50:32,321 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 01:50:32,321 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 01:50:32,321 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:50:32,321 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: shutting down 01:50:32,321 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 01:50:32,321 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 01:50:32,321 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-672051887] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-672051887] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:32,321 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-672051887] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-672051887] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:32,322 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#1068247579] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1068247579] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:32,322 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-672051887] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-672051887] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:32,322 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#1068247579] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1068247579] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:32,322 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-672051887] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-672051887] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:32,322 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:32,322 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:32,328 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:32,328 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:32,328 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:32,328 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:32,329 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:32,341 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 01:50:32,342 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Stopping Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:32,342 PM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Closing 01:50:32,342 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Aborting 0 pending queued transactions 01:50:32,342 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 01:50:32,342 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:32,342 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:33,342 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4449430654510267557] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:33,363 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-2399595631317018924] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:33,365 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:33,365 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:33,366 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:33.368] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:33,376 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:33,376 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:33,377 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:33.378] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:33,389 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:33,394 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3578809270966349067] 01:50:33,394 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:33,395 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:33,395 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:33,396 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:33,396 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:33,404 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:33,404 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:33,404 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:33,407 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:33,411 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [503195975400467503] 01:50:33,411 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:33,413 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:33,413 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:33,424 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:33,424 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:33,426 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:33,434 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-7438649614193870192] 01:50:33,434 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:33,446 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:33,446 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:33,446 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:33,446 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:33,446 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:33,447 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:33,449 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithShardLeaderNotResponding 01:50:33,453 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 01:50:33,453 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#1499642306] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:33,454 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithShardLeaderNotResponding 01:50:33,454 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Creating local shard: member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:33,454 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:33,454 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:33,455 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: no initial generation override, starting from 0 01:50:33,455 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:50:33,455 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@106b714d to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4176f9c7 01:50:33,455 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4746f8d 01:50:33,455 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, 0, 1710165033455, None)) 01:50:33,455 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165033454,0,0)) 01:50:33,455 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4176f9c7 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@494fc130 01:50:33,455 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4b019c46 01:50:33,455 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} started 01:50:33,456 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:33,456 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:33,456 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:33,456 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:33,456 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:33,456 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: updated peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding on shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#1838003980] 01:50:33,456 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithShardLeaderNotResponding, persistent : true 01:50:33,456 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#1838003980] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:33,457 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:33,458 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithShardLeaderNotResponding 01:50:33,461 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithShardLeaderNotResponding with journal batch size 1 01:50:33,462 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-2028945810 created and ready for shard:member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:33,462 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-1:snapshot-read#2010325773] 01:50:33,462 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:33,462 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-2028945810] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:33,462 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-1:snapshot-read#2010325773] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:33,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:33,463 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:33,463 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Initializing shard [cars] 01:50:33,463 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is initialized 01:50:33,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 01:50:33,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 01:50:33,464 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from null to Follower 01:50:33,464 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , registered listener akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding 01:50:33,464 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from null to Follower 01:50:33,465 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:33,467 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 01:50:33,468 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#-1927652467] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:33,468 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: no initial generation override, starting from 0 01:50:33,468 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:50:33,468 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3b7b9ad2 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@68fc43b4 01:50:33,468 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithShardLeaderNotResponding 01:50:33,469 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Creating local shard: member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:33,469 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:33,469 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:33,470 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6224de08 01:50:33,470 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, 0, 1710165033468, None)) 01:50:33,471 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165033467,0,0)) 01:50:33,471 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1bb8021e 01:50:33,471 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@68fc43b4 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6b46c2b0 01:50:33,471 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} started 01:50:33,472 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:33,472 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:33,472 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:33,472 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:33,472 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: updated peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding on shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding#-761616154] 01:50:33,472 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithShardLeaderNotResponding, persistent : true 01:50:33,472 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding#-761616154] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:33,472 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-2090407922 created and ready for shard:member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:33,472 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-2:snapshot-read#-1941387482] 01:50:33,472 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-2090407922] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:33,472 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-2:snapshot-read#-1941387482] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:33,472 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithShardLeaderNotResponding with journal batch size 1 01:50:33,474 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:33,474 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:33,474 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 01:50:33,474 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 01:50:33,474 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from null to Follower 01:50:33,474 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:33,474 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Initializing shard [cars] 01:50:33,474 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding is initialized 01:50:33,475 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithShardLeaderNotResponding , registered listener akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding 01:50:33,475 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-2-shard-cars-testTransactionWithShardLeaderNotResponding from null to Follower 01:50:33,478 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:33,478 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:33,522 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#1499642306] 01:50:33,522 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: findLocalShard : cars 01:50:33,522 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#1838003980] 01:50:33,757 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:33,757 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Received ElectionTimeout - switching to Candidate 01:50:33,757 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithShardLeaderNotResponding] 01:50:33,757 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:33,757 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 1 01:50:33,757 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:33,757 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Closing 01:50:33,757 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 01:50:33,757 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 01:50:33,757 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from Follower to Candidate 01:50:33,757 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from Follower to Candidate 01:50:34,27 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 01:50:34,27 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:34,27 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 2 01:50:34,338 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 01:50:34,338 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:34,338 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 3 01:50:34,408 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:34,408 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:34,409 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:34,409 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:34,409 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:34,409 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:34,409 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:34,409 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:34,409 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:34,417 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:34,417 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:34,417 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:34,417 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:34,417 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:34,418 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:34,453 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:34,647 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 01:50:34,647 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:34,647 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 4 01:50:34,647 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:34,649 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:34,649 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Cluster state: [] 01:50:34,649 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:34,649 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:34,649 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=null 01:50:34,649 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:34,649 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:34,649 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:34,650 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:34,651 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:34,651 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithShardLeaderNotResponding] 01:50:34,651 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:34,651 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:34,651 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: true, hasLeader : true 01:50:34,651 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: became leader with frontend state for [] 01:50:34,651 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from Candidate to Leader 01:50:34,651 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderPayloadVersion=13] 01:50:34,651 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:34,651 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:34,651 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from Candidate to Leader 01:50:34,651 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 01:50:34,651 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Closing 01:50:34,651 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithShardLeaderNotResponding peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:34,652 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding Received follower initial sync status for member-2-shard-cars-testTransactionWithShardLeaderNotResponding status sync done false 01:50:34,652 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithShardLeaderNotResponding, leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderPayloadVersion=13] 01:50:34,652 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:34,652 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:34,652 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 01:50:34,652 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithShardLeaderNotResponding updated: matchIndex: -1, nextIndex: 0 01:50:34,652 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:34,678 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@22536cf4: creating a new snapshot member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 01:50:34,678 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} 01:50:34,678 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:34,678 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:34,679 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#-1927652467] 01:50:34,679 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:34,679 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, history=0, cookie=0} shard 1 01:50:34,679 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 01:50:34,679 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding, primaryVersion=13] 01:50:34,679 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:34,679 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1744715027], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@5be83720 01:50:34,679 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding, primaryVersion=13] 01:50:34,679 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} scheduling timeout in 2 seconds 01:50:34,679 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@2a3da42a, attempting to connect 01:50:34,680 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} is not yet known 01:50:34,680 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, nanosAgo=11490, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:50:34,681 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:50:34,681 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#1838003980], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:50:34,681 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#1838003980], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:34,681 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4393c100 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@60a67eca 01:50:34,681 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:50:34,681 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@60a67eca 01:50:34,681 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1 01:50:34,681 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} 01:50:34,681 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1744715027], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} 01:50:34,681 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:34,681 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1744715027], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1248/0x0000000801301d10@693a8305 01:50:34,682 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#1838003980], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1997340067 nanoseconds 01:50:34,682 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3671745 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1744715027], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=781014130231} 01:50:34,682 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@22536cf4 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#1838003980], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:34,682 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} 01:50:34,682 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@506479cd 01:50:34,682 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7f2c5fa superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3aed5caa, splicing queue 01:50:34,682 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7f2c5fa queue spliced 0 messages 01:50:34,682 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#1838003980], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 682.1 μs 01:50:34,683 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: allocating new open snapshot member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1 01:50:34,683 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithShardLeaderNotResponding_slicer-id-701: slice: identifier: member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0}} 01:50:34,683 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionWithShardLeaderNotResponding_slicer-id-701: Message size: 235 bytes, total slices to send: 1 01:50:34,683 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithShardLeaderNotResponding_slicer-id-701: Message does not need to be sliced - sending original message 01:50:34,684 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1744715027], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0}} 01:50:34,684 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1744715027], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=781014130231, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0}} 01:50:34,684 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0} 01:50:34,684 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 01:50:34,684 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:34,684 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader)_slicer-id-704: Closing 01:50:34,684 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Aborting 0 pending queued transactions 01:50:34,685 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 01:50:34,685 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:34,685 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:34,688 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:34,688 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:34,689 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:34,689 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:34,690 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:34,698 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithShardLeaderNotResponding 01:50:34,698 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@22536cf4: creating a new transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 01:50:34,698 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 01:50:34,698 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:50:34,699 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@22536cf4 readied transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 01:50:34,699 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1744715027], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@19f4bf88 01:50:34,699 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#1838003980], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:34,699 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3671745 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1744715027], modifications=1, protocol=SIMPLE}, enqueuedTicks=781033904326} 01:50:34,699 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-702: slice: identifier: member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1744715027], modifications=1, protocol=SIMPLE}} 01:50:34,699 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-702: Message size: 399 bytes, total slices to send: 1 01:50:34,699 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-702: Message does not need to be sliced - sending original message 01:50:34,700 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:34,700 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:35,703 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-3578809270966349067] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:35,705 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-3578809270966349067] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:36,696 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: running timer on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} 01:50:36,696 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} is empty 01:50:36,696 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: not scheduling timeout on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} 01:50:36,696 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: running timer on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#1838003980], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:36,696 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#1838003980], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2420287 nanoseconds 01:50:36,725 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: running timer on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#1838003980], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:36,725 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: Connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#1838003980], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} timed out entry TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1744715027], modifications=1, protocol=SIMPLE}, enqueuedTicks=781033904326, sessionId=0, txSequence=1} 01:50:36,725 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#1838003980], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} timed out 1 tasks 01:50:36,725 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: not scheduling timeout on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#1838003980], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:36,726 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=FLUSHED} directCommit completed 01:50:36,726 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=FLUSHED}: initiating purge 01:50:36,726 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=DONE} allocated sequence 1 01:50:36,726 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1744715027]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@18ba98d 01:50:36,726 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#1838003980], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1999933309 nanoseconds 01:50:36,726 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3671745 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1744715027]}, enqueuedTicks=783060818072} 01:50:36,726 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:50:36,726 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithShardLeaderNotResponding data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:50:36,726 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 01:50:36,726 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:50:36,726 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithShardLeaderNotResponding data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:50:36,726 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 01:50:36,727 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 01:50:36,727 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:36,727 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Closing 01:50:36,727 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Aborting 0 pending queued transactions 01:50:36,727 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 01:50:36,728 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:36,728 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:37,743 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [503195975400467503] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:37,747 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:37,748 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:37,749 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:37.750] [cluster-test-akka.actor.internal-dispatcher-6] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:37,760 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:37,760 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/11/2024 13:50:37.763] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:37,779 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:37,785 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-2868397738103936171] 01:50:37,785 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:37,786 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:37,786 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:37,786 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:37,786 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:37,795 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:37,795 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:37,795 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:37,797 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:37,801 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-8733147704575464777] 01:50:37,801 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:37,808 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:37,808 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:37,809 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:37,809 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:37,816 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:37,827 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [451298172216894777] 01:50:37,827 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:37,830 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:37,830 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:37,830 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:37,830 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:37,838 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:37,839 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:37,840 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:37,841 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 01:50:37,844 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:37,849 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:50:37,851 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1708380981] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:37,855 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 01:50:37,855 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: no initial generation override, starting from 0 01:50:37,855 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:37,855 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:50:37,855 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@499017b8 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@394259b3 01:50:37,855 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@a554595 01:50:37,855 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:37,855 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:37,855 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1710165037855, None)) 01:50:37,855 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165037854,0,0)) 01:50:37,855 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@52c96d6c 01:50:37,855 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@394259b3 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@70af97d6 01:50:37,855 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} started 01:50:37,856 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:37,857 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:37,859 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 01:50:37,859 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:37,859 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:37,862 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:37,862 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:37,862 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: updated peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182] 01:50:37,862 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 01:50:37,862 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:37,862 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-36527830 created and ready for shard:member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:37,862 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-36527830] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:37,862 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-1335491632] 01:50:37,863 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-1335491632] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:37,864 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#661465975] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:37,864 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:50:37,865 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:37,867 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 01:50:37,868 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:37,868 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:37,868 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:50:37,868 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:37,868 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 01:50:37,869 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 01:50:37,869 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 01:50:37,869 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 01:50:37,869 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: no initial generation override, starting from 0 01:50:37,869 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:50:37,869 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3c46fe70 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@640349c4 01:50:37,869 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1971bfe 01:50:37,869 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 01:50:37,869 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 01:50:37,869 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:37,869 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1710165037869, None)) 01:50:37,870 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:37,870 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165037868,0,0)) 01:50:37,870 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:37,870 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6f4a3348 01:50:37,870 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@640349c4 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4bb24f3d 01:50:37,870 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} started 01:50:37,870 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:37,870 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:37,870 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:37,870 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:37,870 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: updated peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-241279321] 01:50:37,871 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 01:50:37,871 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 01:50:37,871 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-241279321] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:37,871 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-2:snapshot-read#-359879813] 01:50:37,871 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-2:snapshot-read#-359879813] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:37,871 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#303772332 created and ready for shard:member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:37,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:37,872 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:37,872 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 01:50:37,872 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 01:50:37,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:50:37,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:37,872 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 01:50:37,872 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:37,872 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 01:50:37,872 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#303772332] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:37,872 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 01:50:37,873 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:37,920 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1708380981] 01:50:37,920 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 01:50:37,920 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182] 01:50:38,177 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:38,177 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 01:50:38,177 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:50:38,177 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:38,177 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 1 01:50:38,177 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:38,177 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:50:38,177 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:50:38,177 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:38,177 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 01:50:38,177 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 01:50:38,406 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:38,467 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:38,467 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:38,467 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 2 01:50:38,747 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:38,747 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:38,747 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 3 01:50:38,798 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:38,798 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:38,798 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:38,798 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:38,798 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:38,798 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:38,798 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:38,798 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:38,798 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:38,803 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:38,803 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:38,803 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:38,803 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:38,803 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:38,803 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:38,843 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:38,977 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:38,977 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:38,977 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 4 01:50:38,977 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:38,978 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:38,978 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:50:38,978 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:38,978 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:38,978 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=null 01:50:38,978 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:38,978 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:38,978 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:38,979 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:38,979 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:38,980 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:50:38,980 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:38,980 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:38,980 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 01:50:38,980 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [] 01:50:38,980 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Leader 01:50:38,980 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 01:50:38,980 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:38,980 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:38,980 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Leader 01:50:38,980 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 01:50:38,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:38,981 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 01:50:38,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:38,981 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:38,981 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:38,981 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 01:50:38,981 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 01:50:38,981 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: -1, nextIndex: 0 01:50:38,981 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:39,26 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@420ac1f7: creating a new transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:50:39,27 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} 01:50:39,27 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:39,27 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:39,27 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#661465975] 01:50:39,27 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} shard 1 01:50:39,27 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:50:39,27 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:39,27 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:39,27 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 01:50:39,27 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@420ac1f7 readied transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:50:39,27 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 01:50:39,27 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857], modifications=2, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@1a9d2160 01:50:39,28 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} scheduling timeout in 2 seconds 01:50:39,28 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@c996aec, attempting to connect 01:50:39,29 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} is not yet known 01:50:39,29 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, nanosAgo=10460, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:50:39,30 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:50:39,30 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:50:39,30 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:39,30 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@40a39356 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@50d7eb5f 01:50:39,30 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:50:39,30 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@50d7eb5f 01:50:39,30 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:50:39,30 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} 01:50:39,30 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857], modifications=2, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} 01:50:39,30 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:39,30 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857], modifications=2, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2258/0x00000008016bac68@61cd1b78 01:50:39,30 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 01:50:39,30 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5f9a81af transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857], modifications=2, protocol=SIMPLE}, enqueuedTicks=785362767413} 01:50:39,30 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-707: slice: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857], modifications=2, protocol=SIMPLE}} 01:50:39,31 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-707: Message size: 411 bytes, total slices to send: 1 01:50:39,31 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-707: Message does not need to be sliced - sending original message 01:50:39,31 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@420ac1f7 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:39,31 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} 01:50:39,31 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@773de408 01:50:39,31 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@754bce6e superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@4a4b2e6f, splicing queue 01:50:39,31 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@754bce6e queue spliced 0 messages 01:50:39,31 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 928.6 μs 01:50:39,32 PM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: allocating new open transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:50:39,32 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: readying transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:50:39,32 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 ready 01:50:39,32 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transitioned member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 to ready 01:50:39,32 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 initiating direct canCommit 01:50:39,32 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Validating transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:50:39,32 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 validated 01:50:39,32 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:39,32 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 initiating direct preCommit 01:50:39,32 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Preparing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:50:39,32 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 candidate ready 01:50:39,32 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1: canCommit - candidate rootPath: / 01:50:39,32 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1: canCommit - messages: [] 01:50:39,32 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 01:50:39,32 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 prepared 01:50:39,32 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:39,33 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 initiating direct commit 01:50:39,33 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Starting commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:50:39,33 PM [cluster-test-shard-dispatcher-24] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 182 01:50:39,33 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}] 01:50:39,33 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}] 01:50:39,33 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:39,33 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:39,33 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,33 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}]]] 01:50:39,33 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 submitted to persistence 01:50:39,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}]]] 01:50:39,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 01:50:39,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:39,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log Simple{size=182} 01:50:39,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}] 01:50:39,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 1 01:50:39,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:39,35 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}] 01:50:39,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}] 01:50:39,35 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 01:50:39,35 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 0, nextIndex: 1 01:50:39,35 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:39,35 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 0 01:50:39,35 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 01:50:39,35 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 0 data Simple{size=182} 01:50:39,35 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Resuming commit of transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:50:39,36 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@6ebea29b}, purged=MutableUnsignedLongSet{size=0}} 01:50:39,36 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 01:50:39,36 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:50:39,36 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1: commit - successfulFromPrevious: [] 01:50:39,36 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:50:39,36 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#329335880] with name member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 01:50:39,36 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#329335880] 01:50:39,36 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:39,36 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=0 01:50:39,36 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 01:50:39,36 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#329335880] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:39,36 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 96.83 μs 01:50:39,37 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857], modifications=2, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0}} 01:50:39,37 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 0 01:50:39,37 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857], modifications=2, protocol=SIMPLE}, enqueuedTicks=785362767413, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0}} 01:50:39,37 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857], modifications=2, protocol=SIMPLE} completed with TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0} 01:50:39,37 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:39,37 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 0 01:50:39,37 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 01:50:39,37 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} directCommit completed 01:50:39,37 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 01:50:39,37 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:50:39,37 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#12783857]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@24b4fa66 01:50:39,37 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 0 data Simple{size=182} 01:50:39,37 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} 01:50:39,37 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying foreign transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:50:39,37 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SEALED}: initiating purge 01:50:39,37 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} allocated sequence 1 01:50:39,37 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#12783857]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@3cb1575c 01:50:39,37 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:39,37 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5f9a81af transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#12783857]}, enqueuedTicks=785372228679} 01:50:39,37 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4105bc00}, purged=MutableUnsignedLongSet{size=0}} 01:50:39,37 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 01:50:39,37 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:50:39,37 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1412949551] with name member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 01:50:39,37 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1412949551] 01:50:39,37 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 01:50:39,37 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=0 01:50:39,37 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1412949551] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:39,37 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 43.76 μs 01:50:39,38 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: purging transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:50:39,38 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testSingleShardTransactionsWithLeaderChanges 01:50:39,38 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 01:50:39,38 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 01:50:39,38 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:39,38 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:39,38 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,38 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 01:50:39,38 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 01:50:39,38 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Re-initializing to Follower with previous leaderId member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,39 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:50:39,39 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : true 01:50:39,39 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:39,39 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,39 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-709: Closing 01:50:39,39 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 01:50:39,39 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:50:39,40 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:39,40 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:39,43 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:39,43 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:39,43 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:39,43 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:39,58 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 01:50:39,108 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#661465975] 01:50:39,108 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 01:50:39,108 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-241279321] 01:50:39,108 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,158 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,187 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 01:50:39,187 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 01:50:39,187 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 01:50:39,187 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 01:50:39,187 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:50:39,187 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,187 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 5 01:50:39,187 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,187 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:50:39,187 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:50:39,187 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:50:39,187 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:39,187 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:39,188 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 01:50:39,188 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=-1] 01:50:39,188 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:39,188 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 01:50:39,188 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:39,188 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5f9a81af superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@31d0abfc, splicing queue 01:50:39,188 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 01:50:39,188 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5f9a81af queue spliced 1 messages 01:50:39,188 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:39,188 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: refreshing backend for shard 1 01:50:39,188 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:50:39,188 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:39,188 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:39,188 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#661465975] 01:50:39,188 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:39,188 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Candidate, leaderId=null, activeMember=true] 01:50:39,215 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:39,220 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2184304946088414509] 01:50:39,220 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:39,222 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:39,222 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:39,223 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:39,223 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:39,232 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:39,233 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:39,235 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 01:50:39,238 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:50:39,239 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-718418244] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:39,246 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: recovered identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:50:39,253 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 01:50:39,253 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} 01:50:39,253 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@19a55fca to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@40bea042 01:50:39,253 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,253 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6d780ee9 01:50:39,253 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1710165039253, None)) 01:50:39,253 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165039252,0,0)) 01:50:39,253 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@311d3990 01:50:39,253 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@40bea042 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@20915f0d 01:50:39,254 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} started 01:50:39,254 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:39,254 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,255 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 01:50:39,255 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-687212648] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:39,255 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-1192058052 created and ready for shard:member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,255 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-947639655] 01:50:39,255 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-1192058052] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:39,255 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-947639655] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:39,255 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 01:50:39,256 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:50:39,256 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,256 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:50:39,256 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,256 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:50:39,256 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,256 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:50:39,256 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,256 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 01:50:39,256 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received ReplicatedLogEntry for recovery: index: 0, size: 182 01:50:39,256 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 01:50:39,256 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received apply journal entries for recovery, applying to state: 0 to 0 01:50:39,256 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: starting log recovery batch with max size 1 01:50:39,257 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@17ae4f6}, purged=MutableUnsignedLongSet{size=0}} 01:50:39,257 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 01:50:39,257 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:50:39,257 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 01:50:39,257 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 01:50:39,257 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received ReplicatedLogEntry for recovery: index: 1, size: 74 01:50:39,257 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:39,257 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed in 515.5 μs - Switching actor to Follower - last log index = 1, last log term = 4, snapshot index = -1, snapshot term = -1, journal size = 2 01:50:39,257 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 01:50:39,257 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 01:50:39,257 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:50:39,257 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:39,257 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 01:50:39,257 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 01:50:39,257 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 01:50:39,304 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#661465975] 01:50:39,304 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 01:50:39,304 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-241279321] 01:50:39,347 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:39,347 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,347 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 6 01:50:39,347 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received ShardNotInitializedTimeout message for shard cars 01:50:39,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Returning NoShardLeaderException for shard cars 01:50:39,407 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:39,407 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:39,407 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:39,407 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:39,507 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:39,507 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,507 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 7 01:50:39,507 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,677 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:39,677 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,677 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 8 01:50:39,677 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,817 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:39,817 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,817 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 9 01:50:39,817 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,833 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(akka://cluster-test@127.0.0.1:2558, Removed) 01:50:39,833 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 01:50:39,833 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:50:39,833 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:39,833 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 5 01:50:39,833 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:50:39,833 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:50:39,833 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:50:39,833 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:39,833 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 01:50:39,833 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 01:50:40,36 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:40,36 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:40,36 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 10 01:50:40,37 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:40,53 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-2868397738103936171] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:40,56 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [-2868397738103936171] but current is [2184304946088414509]. Reason: ActorSystem terminated 01:50:40,93 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2558, Leaving)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 01:50:40,93 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking unreachable node [akka://cluster-test@127.0.0.1:2558] as [Down] 01:50:40,93 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:40,107 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] LocalActorRefProvider$$anon$2 - Message [akka.cluster.ddata.Replicator$Internal$Status] wrapped in [akka.actor.ActorSelectionMessage] to Actor[akka://cluster-test/] was dropped. Unknown system with UID [451298172216894777]. This system with UID [2184304946088414509] was probably restarted. Messages will be accepted when new handshake has been completed.. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:40,152 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4], my term is 5 01:50:40,152 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 5, votedFor: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastIndex: 1, lastTerm: 4 01:50:40,153 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=5, voteGranted=false] 01:50:40,153 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4], my term is 5 01:50:40,153 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 6 in "RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than Candidate's term 5 - switching to Follower 01:50:40,153 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=null 01:50:40,153 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 1, lastTerm: 4 01:50:40,153 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=6, voteGranted=false] 01:50:40,153 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 6 01:50:40,153 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:50:40,153 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:40,153 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#-2021085064]] intended for system uid [-2868397738103936171], self uid is [2184304946088414509] 01:50:40,153 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 01:50:40,153 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 01:50:40,154 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=false], my term is 10 01:50:40,154 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=false], current voteCount: 1 01:50:40,154 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=false], my term is 10 01:50:40,154 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=false], current voteCount: 1 01:50:40,154 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:40,154 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:50:40,154 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:40,154 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 7 in "RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 6 - updating term 01:50:40,154 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=null 01:50:40,154 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 7, votedFor: null, lastIndex: 1, lastTerm: 4 01:50:40,154 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=7, voteGranted=false] 01:50:40,155 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:40,155 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:50:40,155 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:40,155 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 8 in "RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 7 - updating term 01:50:40,155 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=null 01:50:40,155 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 8, votedFor: null, lastIndex: 1, lastTerm: 4 01:50:40,155 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=8, voteGranted=false] 01:50:40,155 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:40,155 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:50:40,155 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:40,155 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 9 in "RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 8 - updating term 01:50:40,155 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=null 01:50:40,155 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 9, votedFor: null, lastIndex: 1, lastTerm: 4 01:50:40,155 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=9, voteGranted=false] 01:50:40,155 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=7, voteGranted=false], my term is 10 01:50:40,155 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=7, voteGranted=false], current voteCount: 1 01:50:40,155 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=8, voteGranted=false], my term is 10 01:50:40,156 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=8, voteGranted=false], current voteCount: 1 01:50:40,156 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=9, voteGranted=false], my term is 10 01:50:40,156 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:40,156 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=9, voteGranted=false], current voteCount: 1 01:50:40,156 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:50:40,156 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:40,156 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 10 in "RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 9 - updating term 01:50:40,156 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=null 01:50:40,156 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 10, votedFor: null, lastIndex: 1, lastTerm: 4 01:50:40,156 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=10, voteGranted=false] 01:50:40,156 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=10, voteGranted=false], my term is 10 01:50:40,157 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=10, voteGranted=false], current voteCount: 1 01:50:40,234 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2558, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 01:50:40,256 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:40,256 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=11, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:40,256 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 11 01:50:40,256 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=11, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:40,257 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:40,257 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:50:40,257 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=11, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:40,257 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 11 in "RequestVote [term=11, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 10 - updating term 01:50:40,257 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=11, votedFor=null 01:50:40,257 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=11, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 11, votedFor: null, lastIndex: 1, lastTerm: 4 01:50:40,257 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=11, voteGranted=false] 01:50:40,258 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=11, voteGranted=false], my term is 11 01:50:40,258 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=11, voteGranted=false], current voteCount: 1 01:50:40,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:40,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=12, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:40,407 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 12 01:50:40,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=12, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:40,408 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:40,408 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:50:40,408 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=12, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:40,408 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 12 in "RequestVote [term=12, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 11 - updating term 01:50:40,408 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=12, votedFor=null 01:50:40,408 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=12, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 12, votedFor: null, lastIndex: 1, lastTerm: 4 01:50:40,408 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=12, voteGranted=false] 01:50:40,409 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=12, voteGranted=false], my term is 12 01:50:40,409 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=12, voteGranted=false], current voteCount: 1 01:50:40,426 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:40,426 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:40,426 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:40,426 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#661465975] 01:50:40,426 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:40,426 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Candidate, leaderId=null, activeMember=true] 01:50:40,597 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:40,597 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=13, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:40,597 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 13 01:50:40,597 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=13, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:40,598 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:40,598 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:50:40,598 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=13, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:40,598 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 13 in "RequestVote [term=13, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 12 - updating term 01:50:40,598 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=13, votedFor=null 01:50:40,598 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=13, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 13, votedFor: null, lastIndex: 1, lastTerm: 4 01:50:40,599 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=13, voteGranted=false] 01:50:40,599 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=13, voteGranted=false], my term is 13 01:50:40,599 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=13, voteGranted=false], current voteCount: 1 01:50:40,646 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received ShardNotInitializedTimeout message for shard cars 01:50:40,646 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Returning NoShardLeaderException for shard cars 01:50:40,647 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:40,647 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:40,647 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:40,647 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:40,743 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(akka://cluster-test@127.0.0.1:2558, Removed) 01:50:40,743 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 01:50:40,743 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:50:40,743 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=14, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:40,743 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 14 01:50:40,743 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 14 01:50:40,743 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:50:40,743 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:50:40,743 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:40,743 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 01:50:40,743 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 01:50:40,776 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:40,776 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=14, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:40,776 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 14 01:50:40,777 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=14, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:40,778 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=14, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4], my term is 14 01:50:40,778 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=14, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 14, votedFor: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastIndex: 1, lastTerm: 4 01:50:40,778 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=14, voteGranted=false] 01:50:40,779 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=14, voteGranted=false], my term is 14 01:50:40,779 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=14, voteGranted=false], current voteCount: 1 01:50:40,884 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is removing unreachable node [akka://cluster-test@127.0.0.1:2558] 01:50:40,884 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [-2868397738103936171] but current is [2184304946088414509]. Reason: Cluster member removed, previous status [Down] 01:50:40,897 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:40,897 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=15, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:40,897 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 15 01:50:40,897 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=15, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:40,899 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=15, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4], my term is 14 01:50:40,899 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 15 in "RequestVote [term=15, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than Candidate's term 14 - switching to Follower 01:50:40,899 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=15, votedFor=null 01:50:40,899 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=15, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 15, votedFor: null, lastIndex: 1, lastTerm: 4 01:50:40,899 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=15, voteGranted=false] 01:50:40,899 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 15 01:50:40,899 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:50:40,899 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:40,899 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 01:50:40,899 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 01:50:40,900 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=15, voteGranted=false], my term is 15 01:50:40,900 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=15, voteGranted=false], current voteCount: 1 01:50:41,46 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: running timer on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} 01:50:41,46 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} is empty 01:50:41,47 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: not scheduling timeout on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} 01:50:41,47 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: running timer on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:41,47 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} is empty 01:50:41,47 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: not scheduling timeout on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:41,77 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:41,77 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=16, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:41,77 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 16 01:50:41,77 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=16, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:41,78 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:41,78 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:50:41,78 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=16, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:41,78 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 16 in "RequestVote [term=16, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 15 - updating term 01:50:41,78 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=16, votedFor=null 01:50:41,78 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=16, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 16, votedFor: null, lastIndex: 1, lastTerm: 4 01:50:41,78 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=16, voteGranted=false] 01:50:41,79 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=16, voteGranted=false], my term is 16 01:50:41,79 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=16, voteGranted=false], current voteCount: 1 01:50:41,206 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: running timer on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:41,206 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 01:50:41,232 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberRemoved: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:41,232 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [-2868397738103936171] but current is [2184304946088414509]. Reason: Cluster member removed, previous status [Down] 01:50:41,259 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING, roles [member-1, dc-default], version [0.0.0] 01:50:41,277 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:41,277 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=17, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:41,277 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 17 01:50:41,277 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=17, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:41,279 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Welcome from [akka://cluster-test@127.0.0.1:2559] 01:50:41,279 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:41,279 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:50:41,279 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=17, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:41,279 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 17 in "RequestVote [term=17, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 16 - updating term 01:50:41,279 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=17, votedFor=null 01:50:41,279 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=17, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 17, votedFor: null, lastIndex: 1, lastTerm: 4 01:50:41,279 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=17, voteGranted=false] 01:50:41,280 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:41,280 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:41,280 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:41,280 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:41,280 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:41,280 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:41,280 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:41,285 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=17, voteGranted=false], my term is 17 01:50:41,285 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=17, voteGranted=false], current voteCount: 1 01:50:41,417 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:41,417 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=18, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:41,417 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 18 01:50:41,417 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=18, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:41,418 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:41,418 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:50:41,418 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=18, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:41,418 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 18 in "RequestVote [term=18, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 17 - updating term 01:50:41,418 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=18, votedFor=null 01:50:41,418 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=18, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 18, votedFor: null, lastIndex: 1, lastTerm: 4 01:50:41,418 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=18, voteGranted=false] 01:50:41,419 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=18, voteGranted=false], my term is 18 01:50:41,419 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=18, voteGranted=false], current voteCount: 1 01:50:41,433 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Joining), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Joining) 01:50:41,433 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 01:50:41,433 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:50:41,433 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=19, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:41,433 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 19 01:50:41,433 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=19, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:41,434 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 19 01:50:41,434 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:50:41,434 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:50:41,434 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:41,434 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 01:50:41,434 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 01:50:41,434 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=19, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4], my term is 18 01:50:41,434 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 19 in "RequestVote [term=19, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than Candidate's term 18 - switching to Follower 01:50:41,434 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=19, votedFor=null 01:50:41,434 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=19, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 19, votedFor: null, lastIndex: 0, lastTerm: 4 01:50:41,434 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=19, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:41,434 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=19, voteGranted=true] 01:50:41,435 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 19 01:50:41,435 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:50:41,435 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:41,435 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 01:50:41,435 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 01:50:41,435 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=19, voteGranted=true], my term is 19 01:50:41,435 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=19, voteGranted=true], current voteCount: 1 01:50:41,435 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): LastApplied index 0 is behind last index 1 - switching to PreLeader 01:50:41,435 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Election: Leader has following peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:50:41,436 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:41,436 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 19 01:50:41,436 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 01:50:41,436 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges] 01:50:41,436 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=2, term=19, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:50:41,436 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=19, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:50:41,436 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Replicate message: identifier: null, logIndex: 2, payload: class org.opendaylight.controller.cluster.raft.persisted.NoopPayload, isSendImmediate: true 01:50:41,436 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:41,436 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to PreLeader 01:50:41,436 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 01:50:41,436 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:41,436 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:41,436 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to PreLeader 01:50:41,437 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:41,437 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:41,437 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 01:50:41,437 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 01:50:41,437 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:41,437 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:41,437 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=2, term=19, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:50:41,442 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 0, nextIndex: 1 01:50:41,442 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:41,442 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 01:50:41,442 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:41,442 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}], SimpleReplicatedLogEntry [index=2, term=19, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c]]] 01:50:41,443 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:41,443 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}], SimpleReplicatedLogEntry [index=2, term=19, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c]]] 01:50:41,443 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 2 01:50:41,443 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:41,443 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 01:50:41,443 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 01:50:41,443 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c 01:50:41,443 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=19, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:50:41,443 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 3 01:50:41,443 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=19, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=2, logLastTerm=19, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:41,443 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 01:50:41,443 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=2, term=19, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c] 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 2, nextIndex: 3 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 4 does not match the current term 19 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Setting last applied to 1 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: replication of member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 has no callback 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Setting last applied to 2 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 1 to 2 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 1 for term 19 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Received ApplyState [identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}]] - lastApplied: 2, lastIndex: 2 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 19 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader)_slicer-id-712: Closing 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=2 01:50:41,444 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from PreLeader to Leader 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from PreLeader to Leader 01:50:41,444 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 01:50:41,444 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 2 01:50:41,444 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 01:50:41,444 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 1 01:50:41,444 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 01:50:41,444 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 01:50:41,445 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 01:50:41,445 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:50:41,445 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 2 01:50:41,445 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 1 to 2 01:50:41,445 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@76ed877c 01:50:41,445 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 1 for term 19 01:50:41,445 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=2 01:50:41,469 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:41,470 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@420ac1f7: creating a new transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:50:41,470 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:50:41,470 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:50:41,470 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@420ac1f7 readied transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:50:41,470 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@160d925f 01:50:41,470 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} has a successor, not scheduling timer 01:50:41,667 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:41,667 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:41,667 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:41,667 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#661465975] 01:50:41,667 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:41,667 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, activeMember=true] 01:50:41,887 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received ShardNotInitializedTimeout message for shard cars 01:50:41,887 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Returning NoShardLeaderException for shard cars 01:50:41,888 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:41,888 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:41,888 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:41,889 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:41,903 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:42,280 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:42,280 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:42,280 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:42,280 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 01:50:42,757 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 01:50:42,757 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Joining - leader is unavailable 01:50:42,757 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Joining), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 01:50:42,757 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 01:50:42,757 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:50:42,757 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=20, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:42,757 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 20 01:50:42,757 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=20, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=2, lastLogTerm=19] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:42,757 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 20 01:50:42,757 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:50:42,757 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:50:42,757 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:42,757 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:42,757 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 01:50:42,757 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=-1] 01:50:42,758 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:42,758 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 01:50:42,758 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:42,759 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:42,759 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Cluster state: [] 01:50:42,759 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Candidate in requestVote:RequestVote [term=20, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=2, lastLogTerm=19] with higher term appears reachable, updating term. 01:50:42,759 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Term 20 in "RequestVote [term=20, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=2, lastLogTerm=19]" message is greater than leader's term 19 - switching to Follower 01:50:42,759 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=20, votedFor=null 01:50:42,759 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader) :- Switching from behavior Leader to Follower, election term: 20 01:50:42,759 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-713: Closing 01:50:42,759 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:50:42,759 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:42,759 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: removing frontend state for [member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges] 01:50:42,759 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:42,759 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Leader to Follower 01:50:42,759 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=-1] 01:50:42,759 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:42,759 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:42,759 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Leader to Follower 01:50:42,906 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:42,906 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:42,906 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:42,906 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#661465975] 01:50:42,907 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:42,907 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Candidate, leaderId=null, activeMember=true] 01:50:42,911 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:42,911 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:42,911 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:42,911 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized - notifying 1 OnShardInitialized callbacks 01:50:42,911 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:42,916 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:42,916 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=21, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:42,916 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 21 01:50:42,916 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=21, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=2, lastLogTerm=19] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:42,918 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:42,918 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:50:42,918 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=21, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=2, lastLogTerm=19] with higher term appears reachable, updating term. 01:50:42,918 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 21 in "RequestVote [term=21, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=2, lastLogTerm=19]" message is greater than follower's term 20 - updating term 01:50:42,918 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=21, votedFor=null 01:50:42,918 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=21, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=2, lastLogTerm=19] - currentTerm: 21, votedFor: null, lastIndex: 2, lastTerm: 19 01:50:42,918 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=21, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:42,918 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=21, voteGranted=true] 01:50:42,919 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=21, voteGranted=true], my term is 21 01:50:42,919 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=21, voteGranted=true], current voteCount: 1 01:50:42,919 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:50:42,919 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:42,919 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Leader, election term: 21 01:50:42,919 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 01:50:42,919 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges] 01:50:42,919 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Leader 01:50:42,919 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 01:50:42,919 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized - notifying 1 OnShardInitialized callbacks 01:50:42,919 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:42,919 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Leader 01:50:42,919 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is ready - notifying 1 OnShardInitialized callbacks 01:50:42,919 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-241279321, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@352aeb0e] 01:50:42,919 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 01:50:42,919 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:42,919 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-241279321, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@352aeb0e] 01:50:42,919 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@5170515e, attempting to connect 01:50:42,920 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:42,920 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:42,920 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:50:42,920 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 01:50:42,920 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:42,920 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:42,920 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 01:50:42,920 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 01:50:42,920 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-241279321], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:50:42,920 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-241279321], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:42,920 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@50d7eb5f instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3382d463 01:50:42,920 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} previous state DONE 01:50:42,920 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} previous state FLUSHED 01:50:42,920 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3382d463 01:50:42,920 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:50:42,920 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} 01:50:42,920 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Replaying queued request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#12783857]} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} 01:50:42,920 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#12783857]} 01:50:42,920 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN}: initiating purge 01:50:42,920 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} allocated sequence 0 01:50:42,920 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@598e2270 01:50:42,920 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-241279321], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 2 seconds 01:50:42,920 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3d88741a transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857]}, enqueuedTicks=785372228679} 01:50:42,920 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3382d463 01:50:42,920 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 already purged 01:50:42,920 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:50:42,920 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=OPEN} 01:50:42,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857], modifications=1, protocol=SIMPLE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=OPEN} 01:50:42,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857], modifications=1, protocol=SIMPLE} 01:50:42,921 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 2, nextIndex: 3 01:50:42,921 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:42,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SEALED} allocated sequence 0 01:50:42,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@160d925f 01:50:42,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3d88741a transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857], coordinated=false}, enqueuedTicks=787805318338} 01:50:42,921 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: allocating new ready transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:50:42,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@420ac1f7 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-241279321], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:42,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} 01:50:42,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} 01:50:42,921 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 initiating direct canCommit 01:50:42,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@1231bc72 01:50:42,921 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Validating transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:50:42,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6ce7228a superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@10e7fdbc, splicing queue 01:50:42,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6ce7228a queue spliced 0 messages 01:50:42,921 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 validated 01:50:42,921 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:42,921 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 initiating direct preCommit 01:50:42,921 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Preparing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:50:42,921 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-241279321], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 1.250 ms 01:50:42,921 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 candidate ready 01:50:42,921 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1: canCommit - candidate rootPath: / 01:50:42,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857]} with Envelope{sessionId=1, txSequence=0, message=TransactionPurgeResponse{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0}} 01:50:42,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857]}, enqueuedTicks=785372228679, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionPurgeResponse{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0}} 01:50:42,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE}: purge completed 01:50:42,921 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1: canCommit - messages: [] 01:50:42,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3382d463 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} 01:50:42,921 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 01:50:42,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:50:42,921 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 prepared 01:50:42,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@50d7eb5f purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} 01:50:42,921 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:42,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:50:42,921 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 initiating direct commit 01:50:42,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@40a39356 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} 01:50:42,921 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Starting commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:50:42,922 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 01:50:42,922 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=3, term=21, payload=Simple{size=241}] 01:50:42,922 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=21, payload=Simple{size=241}] 01:50:42,922 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:42,922 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 01:50:42,922 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:42,922 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=2, prevLogTerm=19, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=21, payload=Simple{size=241}]]] 01:50:42,922 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 submitted to persistence 01:50:42,922 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 01:50:42,923 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=3, term=21, payload=Simple{size=241}] 01:50:42,923 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=2, prevLogTerm=19, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=21, payload=Simple{size=241}]]] 01:50:42,923 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 01:50:42,923 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:50:42,923 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log Simple{size=241} 01:50:42,923 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=21, payload=Simple{size=241}] 01:50:42,923 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 2 01:50:42,923 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=21, success=true, followerId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=3, logLastTerm=21, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:42,924 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=3, term=21, payload=Simple{size=241}] 01:50:42,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:42,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 3, nextIndex: 4 01:50:42,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 01:50:42,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 3 01:50:42,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 2 to 3 01:50:42,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 3 data Simple{size=241} 01:50:42,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Resuming commit of transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:50:42,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 01:50:42,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:50:42,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1: commit - successfulFromPrevious: [] 01:50:42,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 01:50:42,925 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857], coordinated=false} with Envelope{sessionId=1, txSequence=1, message=TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0}} 01:50:42,925 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857], coordinated=false}, enqueuedTicks=787805318338, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0}} 01:50:42,925 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} directCommit completed 01:50:42,925 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR}: initiating purge 01:50:42,925 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 1 01:50:42,925 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#12783857]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@e5ac844 01:50:42,925 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@420ac1f7: creating a new snapshot member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0 01:50:42,925 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} has a successor, not scheduling timer 01:50:42,925 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 2 for term 21 01:50:42,925 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-655105182], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:50:42,925 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#12783857]} 01:50:42,925 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SEALED}: initiating purge 01:50:42,925 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=DONE} allocated sequence 1 01:50:42,925 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:50:42,925 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#12783857]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@119727d1 01:50:42,925 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-241279321], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:42,925 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=3 01:50:42,925 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3d88741a transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#12783857]}, enqueuedTicks=789259983667} 01:50:42,925 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0 01:50:42,925 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 01:50:42,925 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: purging transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:50:42,925 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} 01:50:42,925 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=4, term=21, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 01:50:42,925 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=21, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 01:50:42,925 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 101.6 μs 01:50:42,925 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:42,925 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Issuing gracefulStop to shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:42,925 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges - waiting on 1 shards 01:50:42,926 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onShutDown 01:50:42,926 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:42,926 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:50:42,926 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 01:50:42,926 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=4, term=21, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 01:50:42,926 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All shards shutdown - sending PoisonPill to self 01:50:42,926 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:50:42,927 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:42,927 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:50:42,927 PM [main] [WARN] ActorUtils - ShardManager for testSingleShardTransactionsWithLeaderChanges data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:50:42,927 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:42,927 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: shutting down 01:50:42,928 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:50:42,928 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:42,928 PM [main] [WARN] ActorUtils - ShardManager for testSingleShardTransactionsWithLeaderChanges data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:50:42,928 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:42,928 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-718418244] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-718418244] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:42,928 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#197477982] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#197477982] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:42,928 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-718418244] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-718418244] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:42,929 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:42,929 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:42,932 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:42,932 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:42,932 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:42,932 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:42,934 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:42,948 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:42,949 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:42,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-714: Closing 01:50:42,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 01:50:42,949 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:50:42,949 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:42,949 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:43,943 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2184304946088414509] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:43,963 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-8733147704575464777] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:43,968 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:43,968 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:43,968 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:43.969] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:43,978 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:43,978 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:43,980 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:43.981] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:44,2 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:44,10 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-5992412717671324621] 01:50:44,10 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:44,12 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:44,12 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:44,16 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:44,16 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:44,24 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:44,24 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:44,24 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:44,27 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:44,31 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-4041858728347056766] 01:50:44,31 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:44,33 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:44,33 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:44,42 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:44,42 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:44,45 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:44,54 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-5169010188057267704] 01:50:44,54 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:44,54 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:44,54 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:44,58 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3382d463 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=OPEN} 01:50:44,58 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@46541b96 01:50:44,58 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3d88741a transmitting entry ConnectionEntry{request=AbortLocalTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#12783857]}, enqueuedTicks=790393745894} 01:50:44,59 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0} 01:50:44,59 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4393c100 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} 01:50:44,59 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:50:44,59 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1744715027], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fb9a8@2825f76f 01:50:44,59 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} has a successor, not scheduling timer 01:50:44,59 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SEALED} allocated sequence 1 01:50:44,59 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1744715027], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x00000008013037f0@2e17014d 01:50:44,59 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3671745 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1744715027], modifications=0, protocol=ABORT}, enqueuedTicks=790394165101} 01:50:44,59 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-702: slice: identifier: member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=3, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1744715027], modifications=0, protocol=ABORT}} 01:50:44,59 PM [Cleaner-1] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-702: Message size: 294 bytes, total slices to send: 1 01:50:44,59 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-702: Message does not need to be sliced - sending original message 01:50:44,59 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0} 01:50:44,61 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:44,61 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:44,68 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:44,69 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:44,69 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:44,70 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:44,70 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 01:50:44,73 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 01:50:44,73 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#374516160] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:44,76 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: no initial generation override, starting from 0 01:50:44,76 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:50:44,76 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@46219df2 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@36393072 01:50:44,76 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3206a26d 01:50:44,77 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1710165044076, None)) 01:50:44,77 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165044075,0,0)) 01:50:44,77 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@36393072 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2f82b475 01:50:44,77 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} started 01:50:44,77 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 01:50:44,77 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:44,77 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:44,77 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:44,77 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:44,77 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7096fe2 01:50:44,77 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:44,77 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:44,78 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:44,78 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:44,78 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-1-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184] 01:50:44,78 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithIsolatedLeader, persistent : true 01:50:44,78 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:44,79 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 01:50:44,79 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:44,79 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#2121226660 created and ready for shard:member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:44,79 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#2121226660] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:44,79 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/shard-member-1:snapshot-read#-140486892] 01:50:44,80 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/shard-member-1:snapshot-read#-140486892] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:44,80 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 01:50:44,83 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1856509866] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:44,84 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:44,84 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:44,84 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:44,84 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 01:50:44,84 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 01:50:44,84 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithIsolatedLeader is initialized 01:50:44,84 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 01:50:44,84 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 01:50:44,84 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 01:50:44,84 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from null to Follower 01:50:44,87 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 01:50:44,87 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:44,87 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:44,87 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:44,87 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:44,87 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: no initial generation override, starting from 0 01:50:44,87 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:50:44,87 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@50a7b61d to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@79e9033b 01:50:44,87 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@a72c2d6 01:50:44,87 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1710165044087, None)) 01:50:44,88 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:44,89 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165044086,0,0)) 01:50:44,89 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@79e9033b to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4fbe3fed 01:50:44,89 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} started 01:50:44,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@55e289a5 01:50:44,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:44,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:44,90 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:44,90 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:44,90 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-1850078579] 01:50:44,90 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithIsolatedLeader, persistent : true 01:50:44,90 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 01:50:44,90 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-1850078579] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:44,91 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#-508691390 created and ready for shard:member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:44,91 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#-508691390] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:44,91 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#102277934] 01:50:44,91 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#102277934] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:44,92 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:44,92 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:44,92 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:44,92 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 01:50:44,92 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 01:50:44,92 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithIsolatedLeader is initialized 01:50:44,92 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:44,92 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 01:50:44,92 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 01:50:44,92 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-2-shard-cars-testTransactionWithIsolatedLeader from null to Follower 01:50:44,139 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#374516160] 01:50:44,139 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: findLocalShard : cars 01:50:44,140 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184] 01:50:44,389 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:44,389 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Received ElectionTimeout - switching to Candidate 01:50:44,389 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 01:50:44,389 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:44,390 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 1 01:50:44,390 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:44,390 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 01:50:44,390 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:44,390 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 01:50:44,390 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Follower to Candidate 01:50:44,390 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Follower to Candidate 01:50:44,696 PM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:44,709 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 01:50:44,709 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:44,709 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 2 01:50:44,959 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 01:50:44,959 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:44,959 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 3 01:50:45,30 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:45,30 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:45,30 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:45,30 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 01:50:45,30 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:45,30 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:45,30 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:45,30 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:45,31 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:45,35 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:45,35 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 01:50:45,35 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:45,35 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:45,35 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:45,35 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:45,72 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:45,220 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 01:50:45,220 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:45,220 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 4 01:50:45,220 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:45,221 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:45,221 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Cluster state: [] 01:50:45,221 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:45,221 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:45,221 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=null 01:50:45,221 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:45,221 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:45,221 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:45,222 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:45,222 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:45,222 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 01:50:45,222 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:45,222 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:45,222 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:50:45,223 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: became leader with frontend state for [] 01:50:45,223 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Candidate to Leader 01:50:45,223 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 01:50:45,223 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:45,223 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:45,223 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Candidate to Leader 01:50:45,223 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 01:50:45,223 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader: Closing 01:50:45,223 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithIsolatedLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:45,223 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 01:50:45,223 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:45,223 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:45,223 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 01:50:45,223 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done false 01:50:45,224 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: -1, nextIndex: 0 01:50:45,224 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:45,245 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4b402833: creating a new transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:50:45,246 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1} 01:50:45,246 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:45,246 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:45,246 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#374516160] 01:50:45,246 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:45,246 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} shard 1 01:50:45,246 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@476a6e6e] 01:50:45,246 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:50:45,246 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:50:45,246 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@476a6e6e] 01:50:45,246 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@693902e9, attempting to connect 01:50:45,247 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4b402833: creating a new transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 01:50:45,247 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} is not yet known 01:50:45,247 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, nanosAgo=10040, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:50:45,247 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 01:50:45,247 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:50:45,247 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4b402833: creating a new transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:50:45,247 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:50:45,247 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=OPEN} allocated sequence 0 01:50:45,247 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:50:45,247 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:50:45,247 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:45,247 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@48cf1ac0 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@17d2f24c 01:50:45,248 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:50:45,248 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} previous state OPEN 01:50:45,248 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} previous state OPEN 01:50:45,248 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@17d2f24c 01:50:45,248 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:50:45,248 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=OPEN} 01:50:45,248 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@17d2f24c 01:50:45,248 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 01:50:45,248 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=OPEN} 01:50:45,248 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@17d2f24c 01:50:45,248 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:50:45,248 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=OPEN} 01:50:45,248 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4b402833 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:45,248 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} 01:50:45,248 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} 01:50:45,248 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} 01:50:45,248 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@6773ae73 01:50:45,248 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1f4578e7 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@36c00738, splicing queue 01:50:45,248 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1f4578e7 queue spliced 0 messages 01:50:45,248 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 539.7 μs 01:50:45,257 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:50:45,258 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:45,258 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 01:50:45,258 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onShutDown 01:50:45,258 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:45,258 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 01:50:45,258 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 01:50:45,258 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 01:50:45,258 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 01:50:45,259 PM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:50:45,259 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request ModifyTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], modifications=1} 01:50:45,259 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4b402833 readied transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:50:45,260 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithIsolatedLeader 01:50:45,260 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#374516160] 01:50:45,260 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: shutting down 01:50:45,260 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:50:45,260 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1856509866] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1856509866] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:45,260 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cd530] to Actor[akka://cluster-test/user/$a#-718868850] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-718868850] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:45,260 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1856509866] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1856509866] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:45,260 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:45,260 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: findLocalShard : cars 01:50:45,260 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184] 01:50:45,260 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184] 01:50:45,260 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 01:50:45,311 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184] 01:50:45,340 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:45,361 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184] 01:50:45,412 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184] 01:50:45,460 PM [cluster-test-shard-dispatcher-23] [WARN] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: At least 1 followers need to be active, Switching member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 01:50:45,460 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 01:50:45,460 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:45,460 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 4 01:50:45,460 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader)_slicer-id-719: Closing 01:50:45,460 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:50:45,460 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Leader to IsolatedLeader 01:50:45,461 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 01:50:45,461 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:45,462 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:45,463 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184] 01:50:45,463 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request ModifyTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], modifications=1} 01:50:45,463 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4b402833 readied transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 01:50:45,463 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=FLUSHED} allocated sequence 0 01:50:45,463 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@1c969750 01:50:45,463 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:50:45,463 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@640785a5 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], coordinated=false}, enqueuedTicks=791798617392} 01:50:45,464 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: not currently active leader, rejecting request Envelope{sessionId=0, txSequence=0, message=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], coordinated=false}}. isLeader: true, isLeaderActive: false,isLeadershipTransferInProgress: false, paused: false 01:50:45,464 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: request Envelope{sessionId=0, txSequence=0, message=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], coordinated=false}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:45,464 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} indicated no leadership, reconnecting it org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:45,464 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@640785a5 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@4a8d9883, splicing queue 01:50:45,464 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:50:45,464 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@640785a5 queue spliced 1 messages 01:50:45,464 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:45,464 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: refreshing backend for shard 1 01:50:45,464 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:50:45,464 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:45,464 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:45,465 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#374516160] 01:50:45,465 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:45,465 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 01:50:45,880 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 01:50:45,880 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 01:50:45,880 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:45,880 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:45,880 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:45,881 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:46,50 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:50:46,900 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:46,900 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:46,900 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:46,900 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#374516160] 01:50:46,900 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:46,900 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 01:50:47,319 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 01:50:47,319 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 01:50:47,320 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:47,320 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:47,320 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:47,320 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:48,339 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:48,339 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:48,339 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:48,339 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#374516160] 01:50:48,339 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:48,339 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 01:50:48,759 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 01:50:48,759 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 01:50:48,759 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:48,759 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:48,760 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:48,760 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:49,780 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:49,780 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:49,780 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:49,780 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#374516160] 01:50:49,780 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:49,780 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 01:50:50,200 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 01:50:50,200 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 01:50:50,200 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:50,200 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:50,201 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:50,201 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:51,219 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:51,219 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:51,219 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:51,219 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#374516160] 01:50:51,220 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:51,220 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 01:50:51,639 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 01:50:51,639 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 01:50:51,640 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:51,640 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:51,640 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:51,640 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:52,659 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:52,659 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:52,659 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:52,659 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#374516160] 01:50:52,659 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:52,659 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 01:50:53,79 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 01:50:53,79 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 01:50:53,80 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:53,80 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:53,80 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:53,80 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:54,100 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:54,100 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:54,100 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:54,100 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#374516160] 01:50:54,100 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:54,100 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 01:50:54,520 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 01:50:54,520 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 01:50:54,520 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:54,520 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:54,520 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:54,521 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:55,540 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:55,540 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:55,540 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:55,540 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#374516160] 01:50:55,540 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:55,540 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 01:50:55,959 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 01:50:55,959 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 01:50:55,959 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:55,960 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:55,960 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:55,960 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:56,980 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:56,980 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:56,980 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:56,980 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#374516160] 01:50:56,980 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:56,980 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 01:50:57,400 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 01:50:57,400 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 01:50:57,400 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:57,400 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:57,400 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:57,401 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:57,464 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithIsolatedLeader 01:50:57,465 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request ModifyTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], modifications=1} 01:50:57,465 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:57,465 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4b402833 readied transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:50:57,465 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 01:50:57,465 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:57,467 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:57,469 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 01:50:57,471 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 01:50:57,471 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: recovered identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:50:57,471 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} 01:50:57,471 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@615f8773 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@200b5891 01:50:57,471 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4211b525 01:50:57,472 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1710165057471, None)) 01:50:57,472 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 01:50:57,472 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:57,472 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165057470,0,0)) 01:50:57,472 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:57,472 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 01:50:57,472 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:57,472 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:57,472 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@200b5891 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4df3d189 01:50:57,472 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:57,472 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:57,472 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3baac067 01:50:57,472 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} started 01:50:57,472 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:57,472 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=FLUSHED} allocated sequence 0 01:50:57,472 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:57,472 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@7faff189 01:50:57,472 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:57,473 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:57,473 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:57,473 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-1367589525] 01:50:57,473 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:57,473 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:57,473 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-2-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-1367589525] 01:50:57,473 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 01:50:57,473 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:57,473 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-3-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-1367589525] 01:50:57,473 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithIsolatedLeader, persistent : true 01:50:57,474 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#320874555] 01:50:57,474 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#1862741269 created and ready for shard:member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:57,474 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 01:50:57,474 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:50:57,475 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=null 01:50:57,475 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:50:57,475 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:57,475 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:57,475 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:57,475 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:57,475 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 01:50:57,475 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 01:50:57,475 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:57,475 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 01:50:57,475 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithIsolatedLeader is initialized 01:50:57,475 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 01:50:57,475 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-2-shard-cars-testTransactionWithIsolatedLeader from null to Follower 01:50:57,580 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader: Closing 01:50:57,580 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithIsolatedLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:57,581 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 01:50:57,581 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:57,581 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:57,581 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 01:50:57,581 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done false 01:50:57,581 PM [cluster-test-shard-dispatcher-24] [WARN] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 12357, lastApplied : -1, commitIndex : -1 01:50:57,581 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - IsolatedLeader member-1-shard-cars-testTransactionWithIsolatedLeader switching from IsolatedLeader to Leader 01:50:57,581 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 01:50:57,581 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:57,581 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 4 01:50:57,581 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader)_slicer-id-720: Closing 01:50:57,581 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:50:57,582 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from IsolatedLeader to Leader 01:50:57,582 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from IsolatedLeader to Leader 01:50:57,582 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 01:50:57,582 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done true 01:50:58,420 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:58,420 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:58,420 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:58,420 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#374516160] 01:50:58,420 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:58,420 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@476a6e6e] 01:50:58,420 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@476a6e6e] 01:50:58,421 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@48a74dba, attempting to connect 01:50:58,421 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:50:58,421 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:50:58,421 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:58,422 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@17d2f24c instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@29029c94 01:50:58,422 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:50:58,422 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} previous state FLUSHED 01:50:58,422 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} previous state SEALED 01:50:58,422 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@29029c94 01:50:58,422 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:50:58,422 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=OPEN} 01:50:58,422 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Replaying queued request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], coordinated=false} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=OPEN} 01:50:58,422 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SEALED} allocated sequence 0 01:50:58,422 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@7faff189 01:50:58,422 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:50:58,422 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7235918e transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], coordinated=false}, enqueuedTicks=803807704254} 01:50:58,422 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@29029c94 01:50:58,422 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 01:50:58,422 PM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: allocating new ready transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:50:58,422 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=OPEN} 01:50:58,422 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Replaying queued request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], coordinated=false} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=OPEN} 01:50:58,422 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 initiating direct canCommit 01:50:58,422 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:50:58,422 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SEALED} allocated sequence 0 01:50:58,422 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 validated 01:50:58,422 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@1c969750 01:50:58,422 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:58,422 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7235918e transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], coordinated=false}, enqueuedTicks=791798617392} 01:50:58,422 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 initiating direct preCommit 01:50:58,422 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Preparing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:50:58,422 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@29029c94 01:50:58,423 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:50:58,423 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=OPEN} 01:50:58,423 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 candidate ready 01:50:58,423 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=OPEN} 01:50:58,423 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1: canCommit - candidate rootPath: / 01:50:58,423 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], modifications=1} 01:50:58,423 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1: canCommit - messages: [] 01:50:58,423 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 01:50:58,423 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 prepared 01:50:58,423 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:58,423 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 initiating direct commit 01:50:58,423 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Starting commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:50:58,423 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4b402833 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:58,423 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} 01:50:58,423 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} 01:50:58,423 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} 01:50:58,423 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@6f0f6b1e 01:50:58,423 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@355eecb0 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3887566b, splicing queue 01:50:58,423 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@355eecb0 queue spliced 0 messages 01:50:58,423 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 1.551 ms 01:50:58,423 PM [cluster-test-shard-dispatcher-24] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 171 01:50:58,423 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 01:50:58,423 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 01:50:58,423 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:58,423 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:58,423 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:58,424 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}]]] 01:50:58,424 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 submitted to persistence 01:50:58,424 PM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: allocating new ready transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 01:50:58,424 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 initiating direct canCommit 01:50:58,424 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 01:50:58,424 PM [cluster-test-shard-dispatcher-24] [WARN] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Store Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1: Conflicting modification for path /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars. 01:50:58,424 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to FAILED 01:50:58,424 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], coordinated=false} with Envelope{sessionId=1, txSequence=1, message=TransactionFailure{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: CanCommit failed}} 01:50:58,424 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], coordinated=false}, enqueuedTicks=791798617392, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionFailure{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: CanCommit failed}} 01:50:58,424 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} directCommit completed 01:50:58,424 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR}: initiating purge 01:50:58,424 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 1 01:50:58,424 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1514040777]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@5112e4d7 01:50:58,424 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:58,424 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: CanCommit failed at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction$8.onFailure(FrontendReadWriteTransaction.java:441) at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.failedCanCommit(SimpleShardDataTreeCohort.java:175) at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$4(ShardDataTree.java:862) at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:884) at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:831) at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:986) at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:99) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.directCommit(FrontendReadWriteTransaction.java:433) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleCommitLocalTransaction(FrontendReadWriteTransaction.java:524) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:193) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: OptimisticLockFailedException{message=Optimistic lock failed for path /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, errorList=[RpcError [message=Optimistic lock failed for path /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, severity=ERROR, errorType=APPLICATION, tag=resource-denied, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.tree.api.ConflictingModificationAppliedException: Node was created by other transaction.]]} at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$4(ShardDataTree.java:846) ... 41 more Caused by: org.opendaylight.yangtools.yang.data.tree.api.ConflictingModificationAppliedException: Node was created by other transaction. at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.checkConflicting(SchemaAwareApplyOperation.java:74) at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.checkWriteApplicable(SchemaAwareApplyOperation.java:181) at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:111) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:409) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:368) at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:110) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractDataTreeTip.validate(AbstractDataTreeTip.java:31) at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$4(ShardDataTree.java:838) ... 41 more 01:50:58,425 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 01:50:58,425 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}]]] 01:50:58,425 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 1 01:50:58,425 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:58,425 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log Simple{size=171} 01:50:58,425 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 01:50:58,425 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 1 01:50:58,425 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:58,426 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 01:50:58,426 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 0, nextIndex: 1 01:50:58,426 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:58,426 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 0 01:50:58,426 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from -1 to 0 01:50:58,426 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 0 data Simple{size=171} 01:50:58,426 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Resuming commit of transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:50:58,426 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionWithIsolatedLeader: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@d2cdb44}, purged=MutableUnsignedLongSet{size=0}} 01:50:58,426 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 01:50:58,426 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Committed transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:50:58,426 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1: commit - successfulFromPrevious: [] 01:50:58,426 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:50:58,426 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionWithIsolatedLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-1866824548] with name member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher 01:50:58,426 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:58,427 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=0 01:50:58,427 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-1866824548] 01:50:58,427 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 01:50:58,427 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-1866824548] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:58,427 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 77.65 μs 01:50:58,424 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:58,428 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1514040777]} 01:50:58,428 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SEALED}: initiating purge 01:50:58,428 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=DONE} allocated sequence 1 01:50:58,428 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1514040777]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@45d12270 01:50:58,428 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7235918e transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1514040777]}, enqueuedTicks=804759576584} 01:50:58,428 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], coordinated=false} with Envelope{sessionId=1, txSequence=0, message=TransactionCommitSuccess{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0}} 01:50:58,428 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 01:50:58,428 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], coordinated=false}, enqueuedTicks=803807704254, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCommitSuccess{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0}} 01:50:58,428 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} directCommit completed 01:50:58,428 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 01:50:58,428 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 01:50:58,428 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 01:50:58,428 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:50:58,428 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1514040777]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@7286d16 01:50:58,428 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:58,428 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:58,428 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:58,428 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1514040777]} 01:50:58,428 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 0 01:50:58,428 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SEALED}: initiating purge 01:50:58,428 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=DONE} allocated sequence 1 01:50:58,428 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1514040777]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@59e5796 01:50:58,428 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7235918e transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1514040777]}, enqueuedTicks=804763598869} 01:50:58,429 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:50:58,429 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 0 01:50:58,429 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from -1 to 0 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 01:50:58,429 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 0 data Simple{size=171} 01:50:58,429 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=FLUSHED} allocated sequence 0 01:50:58,429 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x0000000801302470@753912b7 01:50:58,429 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7235918e transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], coordinated=false}, enqueuedTicks=804763976785} 01:50:58,429 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying foreign transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:58,429 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionWithIsolatedLeader: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@6d9acc09}, purged=MutableUnsignedLongSet{size=0}} 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 01:50:58,429 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 01:50:58,429 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Committed transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: allocating new ready transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 initiating direct canCommit 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 validated 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 initiating direct preCommit 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Preparing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:50:58,429 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionWithIsolatedLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-1906784671] with name member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 candidate ready 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1: canCommit - candidate rootPath: / 01:50:58,429 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-1906784671] 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1: canCommit - messages: [] 01:50:58,429 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=0 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1: preCommit - successfulFromPrevious: [] 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 prepared 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 initiating direct commit 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Starting commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:50:58,429 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 69 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=69}] 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=69}] 01:50:58,429 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:58,430 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:50:58,430 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:58,430 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}], SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=69}]]] 01:50:58,430 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 66.38 μs 01:50:58,430 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 submitted to persistence 01:50:58,430 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 01:50:58,433 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:58,433 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=69}] 01:50:58,433 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}], SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=69}]]] 01:50:58,433 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 3 01:50:58,433 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:58,433 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64} 01:50:58,433 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 01:50:58,433 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63} 01:50:58,433 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 01:50:58,433 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log Simple{size=69} 01:50:58,433 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=69}] 01:50:58,433 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 4 01:50:58,433 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:58,434 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 01:50:58,434 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 01:50:58,434 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=69}] 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 3, nextIndex: 4 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 0 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 1 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 0 to 1 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64} 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: replication of member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@418f156f 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: finished purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 2 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 1 to 2 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63} 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: replication of member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@1732fc3b 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: finished purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 3 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 2 to 3 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 3 data Simple{size=69} 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Resuming commit of transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Committed transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1: commit - successfulFromPrevious: [] 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 changing state from COMMIT_PENDING to COMMITTED 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 2 for term 4 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:50:58,435 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=3 01:50:58,435 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1514040777]} with Envelope{sessionId=1, txSequence=2, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1}} 01:50:58,435 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1514040777]}, enqueuedTicks=804759576584, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1}} 01:50:58,435 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=DONE}: purge completed 01:50:58,435 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@29029c94 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=DONE} 01:50:58,435 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 01:50:58,435 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR}: purge completed 01:50:58,435 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@17d2f24c purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} 01:50:58,435 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1514040777]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1}} 01:50:58,435 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1514040777]}, enqueuedTicks=804763598869, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1}} 01:50:58,435 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=DONE}: purge completed 01:50:58,435 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 28.61 μs 01:50:58,435 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@29029c94 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=DONE} 01:50:58,435 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:50:58,435 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@17d2f24c purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} 01:50:58,435 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], coordinated=false} with Envelope{sessionId=1, txSequence=4, message=TransactionCommitSuccess{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0}} 01:50:58,435 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1514040777], coordinated=false}, enqueuedTicks=804763976785, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=TransactionCommitSuccess{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0}} 01:50:58,436 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=FLUSHED} directCommit completed 01:50:58,436 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=FLUSHED}: initiating purge 01:50:58,436 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=DONE} allocated sequence 1 01:50:58,436 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1514040777]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@bf08300 01:50:58,436 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:58,436 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7235918e transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1514040777]}, enqueuedTicks=804770791986} 01:50:58,436 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:50:58,436 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 01:50:58,436 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 01:50:58,436 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:50:58,436 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 01:50:58,436 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:50:58,437 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:58,437 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 01:50:58,437 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onShutDown 01:50:58,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 3 01:50:58,437 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Initiating leader transfer 01:50:58,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 0, lastIndex(): 3 01:50:58,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 1 01:50:58,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 0 to 1 01:50:58,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64} 01:50:58,437 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderId=null, leaderPayloadVersion=13] 01:50:58,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 01:50:58,437 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@753392c 01:50:58,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 01:50:58,437 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:58,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 2 01:50:58,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 1 to 2 01:50:58,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63} 01:50:58,437 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@753392c 01:50:58,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 01:50:58,437 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:50:58,437 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Attempting to transfer leadership 01:50:58,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:50:58,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 3 01:50:58,437 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 2 01:50:58,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 2 to 3 01:50:58,437 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:58,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 3 data Simple{size=69} 01:50:58,437 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7235918e superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@fcf06a6, splicing queue 01:50:58,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying foreign transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:50:58,437 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 01:50:58,437 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:50:58,437 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7235918e queue spliced 1 messages 01:50:58,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 01:50:58,437 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:50:58,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Committed transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:50:58,437 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: refreshing backend for shard 1 01:50:58,437 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1311735184], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:50:58,437 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:50:58,437 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:50:58,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 2 for term 4 01:50:58,437 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 01:50:58,437 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#374516160] 01:50:58,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=3 01:50:58,437 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 28.88 μs 01:50:58,437 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:58,438 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 20000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 01:50:58,438 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:58,438 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionWithIsolatedLeader, matchIndex: 3, lastIndex: 4, isVoting: true 01:50:58,438 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 01:50:58,438 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 1 01:50:58,438 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 01:50:58,438 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64} 01:50:58,438 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 01:50:58,438 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 2 01:50:58,438 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:58,439 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 01:50:58,439 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader] 01:50:58,439 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=null, leaderPayloadVersion=13] 01:50:58,439 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:58,439 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:58,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 4, nextIndex: 5 01:50:58,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 01:50:58,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 4 01:50:58,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 3 to 4 01:50:58,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64} 01:50:58,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: replication of member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@797d1823 01:50:58,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: finished purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:50:58,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 01:50:58,439 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1}} not found in inflight queue, checking pending queue 01:50:58,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:50:58,439 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1514040777]} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1}} 01:50:58,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 3 for term 4 01:50:58,439 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1514040777]}, enqueuedTicks=804770791986, sessionId=1, txSequence=5} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1}} 01:50:58,439 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=DONE}: purge completed 01:50:58,439 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@29029c94 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=DONE} 01:50:58,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:50:58,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionWithIsolatedLeader, matchIndex: 4, lastIndex: 4, isVoting: true 01:50:58,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Follower's log matches - sending ElectionTimeout 01:50:58,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:50:58,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Leader transfer complete 01:50:58,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=4 01:50:58,440 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 4 01:50:58,440 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4 01:50:58,440 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 4 01:50:58,440 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 3 to 4 01:50:58,440 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64} 01:50:58,440 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 01:50:58,440 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:50:58,440 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 3 for term 4 01:50:58,440 PM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:58,440 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=4 01:51:00,463 PM [cluster-test-shard-dispatcher-23] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testTransactionWithIsolatedLeader: Successfully transferred leadership to member-1-shard-cars-testTransactionWithIsolatedLeader in 2.026 s 01:51:00,463 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: leader transfer succeeded - sending PoisonPill 01:51:00,464 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithIsolatedLeader 01:51:00,464 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader)_slicer-id-723: Closing 01:51:00,464 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 01:51:00,464 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 01:51:00,464 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 01:51:00,464 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:51:00,464 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:51:00,465 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:51:00,465 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: shutting down 01:51:00,465 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:51:00,466 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:00,466 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:00,469 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:00,469 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:00,469 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:00,469 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:00,471 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:51:00,481 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:51:00,481 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithIsolatedLeader 01:51:00,481 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 01:51:00,481 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 01:51:00,481 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 01:51:00,481 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:00,482 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:01,483 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-5992412717671324621] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:01,498 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:01,498 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:01,501 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:01.502] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:01,502 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-4041858728347056766] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:01,519 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:01,519 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:01,521 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:01.522] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:01,546 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:51:01,551 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-7269894993774517783] 01:51:01,551 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:51:01,553 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:51:01,553 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:51:01,553 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:01,553 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:01,562 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:51:01,562 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:51:01,562 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:51:01,564 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:51:01,568 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [8839479926786995824] 01:51:01,568 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:51:01,570 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:01,570 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:51:01,575 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:01,575 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:01,583 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:01,591 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [3747711269843975714] 01:51:01,592 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:51:01,593 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:01,593 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:51:01,594 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:01,594 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:01,604 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:01,605 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:01,606 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithMultipleShards 01:51:01,619 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#45888957] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,620 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:51:01,621 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithMultipleShards 01:51:01,622 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:01,624 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithMultipleShards 01:51:01,624 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,624 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: no initial generation override, starting from 0 01:51:01,624 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:51:01,624 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@229e1881 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@408d9ac7 01:51:01,624 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,624 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@58f8d68a 01:51:01,624 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:01,624 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,624 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,624 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testWriteTransactionWithMultipleShards, 0, 1710165061624, None)) 01:51:01,624 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165061623,0,0)) 01:51:01,625 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3e146084 01:51:01,625 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@408d9ac7 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4d2c6cf9 01:51:01,625 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} started 01:51:01,625 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:01,627 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:01,629 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithMultipleShards 01:51:01,630 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:01,630 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,630 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,630 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,630 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards on shard member-1-shard-cars-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#718980146] 01:51:01,630 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,631 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,631 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,631 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards on shard member-1-shard-people-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1398213343] 01:51:01,631 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#718980146] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,631 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-1-shard-cars-testWriteTransactionWithMultipleShards, persistent : true 01:51:01,631 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-people-testWriteTransactionWithMultipleShards, persistent : true 01:51:01,631 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1398213343] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,631 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-909889246] 01:51:01,631 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-notifier#750252189 created and ready for shard:member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,631 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-notifier#750252189] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,631 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-1-shard-cars-testWriteTransactionWithMultipleShards with journal batch size 1 01:51:01,631 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-909889246] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,633 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:01,633 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithMultipleShards 01:51:01,633 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:01,633 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-894855234] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,633 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:01,633 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 01:51:01,633 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [cars] 01:51:01,633 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-1-shard-cars-testWriteTransactionWithMultipleShards is initialized 01:51:01,633 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-people-testWriteTransactionWithMultipleShards with journal batch size 1 01:51:01,633 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-notifier#498043796 created and ready for shard:member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,633 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#1807033553] 01:51:01,634 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-notifier#498043796] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,634 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:01,634 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:01,634 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:01,634 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 01:51:01,641 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from null to Follower 01:51:01,641 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from null to Follower 01:51:01,641 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 01:51:01,643 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: no initial generation override, starting from 0 01:51:01,643 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:51:01,643 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithMultipleShards 01:51:01,643 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3cbfdd16 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@19068e7e 01:51:01,643 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1caff08b 01:51:01,643 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,643 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithMultipleShards, 0, 1710165061643, None)) 01:51:01,643 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:01,643 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165061642,0,0)) 01:51:01,643 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:01,643 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,643 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,643 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@19068e7e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@181e66b3 01:51:01,643 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@31bd05d 01:51:01,644 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} started 01:51:01,644 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:01,644 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,646 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:51:01,647 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:01,653 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [people] 01:51:01,653 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-1-shard-people-testWriteTransactionWithMultipleShards is initialized 01:51:01,653 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from null to Follower 01:51:01,653 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 01:51:01,653 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from null to Follower 01:51:01,656 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#1807033553] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,657 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,657 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,657 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards on shard member-2-shard-cars-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards#-978250488] 01:51:01,657 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:01,657 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithMultipleShards, persistent : true 01:51:01,658 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards#-978250488] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,658 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#1809770595] 01:51:01,658 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:01,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards on shard member-2-shard-people-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards#1532623536] 01:51:01,659 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-people-testWriteTransactionWithMultipleShards, persistent : true 01:51:01,659 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-notifier#1856121641 created and ready for shard:member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,659 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-notifier#1635200045 created and ready for shard:member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:01,660 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-2008069657] 01:51:01,660 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-notifier#1856121641] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,660 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-people-testWriteTransactionWithMultipleShards with journal batch size 1 01:51:01,660 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#1809770595] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,660 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards#1532623536] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,660 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-notifier#1635200045] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,660 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-2008069657] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,664 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:01,664 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:01,664 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:01,664 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 01:51:01,664 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [people] 01:51:01,664 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-testWriteTransactionWithMultipleShards is initialized 01:51:01,664 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,664 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithMultipleShards , received role change from null to Follower 01:51:01,664 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 01:51:01,664 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-2-shard-people-testWriteTransactionWithMultipleShards from null to Follower 01:51:01,665 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithMultipleShards with journal batch size 1 01:51:01,665 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:01,665 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:01,666 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [cars] 01:51:01,666 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithMultipleShards is initialized 01:51:01,666 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:01,666 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 01:51:01,666 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithMultipleShards , received role change from null to Follower 01:51:01,666 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-1-shard-cars-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,666 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 01:51:01,666 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-2-shard-cars-testWriteTransactionWithMultipleShards from null to Follower 01:51:01,694 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#45888957] 01:51:01,694 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: findLocalShard : cars 01:51:01,694 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#718980146] 01:51:01,918 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:01,918 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:51:01,918 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testWriteTransactionWithMultipleShards] 01:51:01,918 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,918 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 01:51:01,918 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:01,918 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Closing 01:51:01,918 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:01,918 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 01:51:01,918 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from Follower to Candidate 01:51:01,918 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from Follower to Candidate 01:51:01,938 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:01,938 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:51:01,938 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testWriteTransactionWithMultipleShards] 01:51:01,938 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,938 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 01:51:01,938 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:01,938 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Closing 01:51:01,938 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:01,938 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 01:51:01,938 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from Follower to Candidate 01:51:01,938 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from Follower to Candidate 01:51:02,238 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:51:02,238 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:02,238 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 01:51:02,244 PM [cluster-test-shard-dispatcher-26] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:02,258 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:51:02,258 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,258 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 01:51:02,273 PM [cluster-test-shard-dispatcher-26] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:02,508 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:51:02,508 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:02,508 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 01:51:02,528 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:51:02,528 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,528 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 01:51:02,568 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:51:02,568 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:51:02,569 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:02,569 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,569 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,569 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards 01:51:02,569 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:02,569 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:02,569 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,569 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,569 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:02,569 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:02,569 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-2-shard-cars-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,569 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-people-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:02,573 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:02,573 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,573 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,573 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards 01:51:02,573 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:02,573 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:02,573 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,573 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,573 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:02,573 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:02,612 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:51:02,748 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:51:02,748 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:02,748 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 01:51:02,748 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:02,749 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:02,749 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Cluster state: [] 01:51:02,749 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:02,749 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:02,749 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 01:51:02,749 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:02,749 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:02,750 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:02,751 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:02,751 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:02,751 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testWriteTransactionWithMultipleShards] 01:51:02,751 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:02,751 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:02,751 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:51:02,751 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: became leader with frontend state for [] 01:51:02,751 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from Candidate to Leader 01:51:02,751 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:51:02,751 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:02,751 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:02,751 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from Candidate to Leader 01:51:02,752 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithMultipleShards: Closing 01:51:02,752 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:02,752 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testWriteTransactionWithMultipleShards status sync done false 01:51:02,752 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:51:02,752 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:51:02,752 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:02,752 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:02,752 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:02,848 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:51:02,848 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,848 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 01:51:02,848 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,849 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:02,849 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Cluster state: [] 01:51:02,849 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:02,849 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:02,849 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 01:51:02,849 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:02,849 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,849 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:02,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:02,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:02,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testWriteTransactionWithMultipleShards] 01:51:02,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:02,855 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:02,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:51:02,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: became leader with frontend state for [] 01:51:02,856 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from Candidate to Leader 01:51:02,856 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:51:02,856 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:02,856 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:02,856 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithMultipleShards: Closing 01:51:02,856 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from Candidate to Leader 01:51:02,856 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,856 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All Shards are ready - data store testWriteTransactionWithMultipleShards is ready 01:51:02,856 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithMultipleShards status sync done false 01:51:02,856 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:51:02,856 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:02,856 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:02,856 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All Shards are ready - data store testWriteTransactionWithMultipleShards is ready 01:51:02,857 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:51:02,857 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:02,869 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testWriteTransactionWithMultipleShards status sync done true 01:51:02,905 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#45888957] 01:51:02,905 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: findLocalShard : people 01:51:02,906 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1398213343] 01:51:02,906 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@516a69de: creating a new transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,906 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} 01:51:02,906 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:02,906 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:02,906 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-894855234] 01:51:02,906 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:02,906 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} shard 1 01:51:02,906 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards, primaryVersion=13] 01:51:02,906 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,906 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:02,906 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards, primaryVersion=13] 01:51:02,907 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} 01:51:02,907 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:51:02,907 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:51:02,907 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-894855234] 01:51:02,907 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} shard 2 01:51:02,907 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,907 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:51:02,907 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@516a69de readied transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,907 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@425716ea 01:51:02,907 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:51:02,907 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801307958@22542213 01:51:02,907 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} scheduling timeout in 30 seconds 01:51:02,909 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:51:02,909 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@13a6da12, attempting to connect 01:51:02,909 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards, primaryVersion=13] 01:51:02,909 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards, primaryVersion=13] 01:51:02,909 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@69bffc61, attempting to connect 01:51:02,911 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} is not yet known 01:51:02,911 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} is not yet known 01:51:02,911 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, nanosAgo=12670, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:51:02,911 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, nanosAgo=12440, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:51:02,912 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:02,912 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:02,912 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#718980146], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:02,912 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#718980146], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:02,912 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6cb48f2d instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7bd4060 01:51:02,912 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:51:02,912 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7bd4060 01:51:02,912 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:02,912 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} 01:51:02,912 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} 01:51:02,912 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:02,912 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2219/0x00000008016a6000@7f79be95 01:51:02,912 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#718980146], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:02,912 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@48c1af5 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=809242236012} 01:51:02,912 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-727: slice: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], modifications=1, protocol=THREE_PHASE}} 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-727: Message size: 390 bytes, total slices to send: 1 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-727: Message does not need to be sliced - sending original message 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@516a69de finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#718980146], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@5852bf0c 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4f9102b4 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@133ef395, splicing queue 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4f9102b4 queue spliced 0 messages 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#718980146], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 691.2 μs 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1398213343], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1398213343], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3068f389 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@57711114 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} previous state FLUSHED 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@57711114 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2219/0x00000008016a6000@74f4d624 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1398213343], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d77bd70 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=809242315604} 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-727: slice: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], modifications=1, protocol=THREE_PHASE}} 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-727: Message size: 395 bytes, total slices to send: 1 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-727: Message does not need to be sliced - sending original message 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@516a69de finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1398213343], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@53ac4836 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@557e8880 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@22616e07, splicing queue 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@557e8880 queue spliced 0 messages 01:51:02,913 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1398213343], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 411.3 μs 01:51:02,914 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithMultipleShards: allocating new open transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:02,914 PM [cluster-test-shard-dispatcher-25] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testWriteTransactionWithMultipleShards: allocating new open transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:02,914 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:02,915 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 ready 01:51:02,915 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: transitioned member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 to ready 01:51:02,915 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating canCommit 01:51:02,915 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:02,915 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 validated 01:51:02,915 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:02,915 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:02,915 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 ready 01:51:02,915 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: transitioned member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 to ready 01:51:02,915 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating canCommit 01:51:02,915 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed canCommit 01:51:02,915 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:02,915 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 validated 01:51:02,915 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:02,915 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed canCommit 01:51:02,916 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0}} 01:51:02,916 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=809242236012, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0}} 01:51:02,916 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 01:51:02,916 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0}} 01:51:02,916 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=809242315604, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0}} 01:51:02,916 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} canCommit completed 01:51:02,916 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:51:02,916 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#349046731]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@43d6db30 01:51:02,916 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} 01:51:02,916 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} allocated sequence 1 01:51:02,916 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#349046731]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1242/0x0000000801303c28@39cc65e6 01:51:02,916 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#718980146], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:02,916 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@48c1af5 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#349046731]}, enqueuedTicks=809251125196} 01:51:02,916 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 01:51:02,916 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#349046731]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013051c0@3a8b9c76 01:51:02,916 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} 01:51:02,916 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} allocated sequence 1 01:51:02,916 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#349046731]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1242/0x0000000801303c28@54f9a66d 01:51:02,916 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1398213343], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:02,916 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d77bd70 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#349046731]}, enqueuedTicks=809251241547} 01:51:02,917 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating preCommit 01:51:02,917 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:02,917 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating preCommit 01:51:02,917 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:02,917 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 candidate ready 01:51:02,917 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2: canCommit - candidate rootPath: / 01:51:02,917 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 candidate ready 01:51:02,917 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1: canCommit - candidate rootPath: / 01:51:02,917 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2: canCommit - messages: [] 01:51:02,917 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2: preCommit - successfulFromPrevious: [] 01:51:02,917 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 prepared 01:51:02,917 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:02,917 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1: canCommit - messages: [] 01:51:02,917 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed preCommit 01:51:02,917 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 01:51:02,917 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 prepared 01:51:02,917 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:02,917 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed preCommit 01:51:02,918 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#349046731]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1}} 01:51:02,918 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#349046731]}, enqueuedTicks=809251241547, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1}} 01:51:02,918 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:51:02,918 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#349046731]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1}} 01:51:02,918 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#349046731]}, enqueuedTicks=809251125196, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1}} 01:51:02,918 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:51:02,918 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:51:02,918 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#349046731]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@7b3a4a1a 01:51:02,918 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} 01:51:02,918 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} allocated sequence 2 01:51:02,918 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#349046731]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@7b3a4a1a 01:51:02,918 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#718980146], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:02,918 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@48c1af5 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#349046731]}, enqueuedTicks=809253404013} 01:51:02,918 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 2 01:51:02,918 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#349046731]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@1d340127 01:51:02,918 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} 01:51:02,918 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} allocated sequence 2 01:51:02,918 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#349046731]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801302000@1d340127 01:51:02,918 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1398213343], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:02,918 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d77bd70 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#349046731]}, enqueuedTicks=809253502584} 01:51:02,924 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating commit 01:51:02,924 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:02,924 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 176 01:51:02,924 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 01:51:02,924 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 01:51:02,924 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:02,925 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:02,925 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,925 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}]]] 01:51:02,925 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 submitted to persistence 01:51:02,925 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating commit 01:51:02,925 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:02,925 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 180 01:51:02,925 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:51:02,925 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:51:02,925 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:02,925 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:02,925 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:02,925 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 01:51:02,925 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 submitted to persistence 01:51:02,925 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 01:51:02,925 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:51:02,927 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}]]] 01:51:02,927 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:51:02,927 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 01:51:02,927 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:02,927 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:51:02,927 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=176} 01:51:02,927 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:02,928 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 01:51:02,928 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=180} 01:51:02,928 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:51:02,928 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Log size is now 1 01:51:02,928 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Log size is now 1 01:51:02,928 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:02,928 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:02,928 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 01:51:02,928 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:51:02,928 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithMultipleShards status sync done true 01:51:02,928 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:51:02,928 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:51:02,928 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:51:02,928 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:51:02,928 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 01:51:02,928 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 01:51:02,928 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:51:02,928 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:51:02,928 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=176} 01:51:02,928 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=180} 01:51:02,928 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:02,928 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:02,928 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendMetadata - member-1-shard-cars-testWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@7166c14d}, purged=MutableUnsignedLongSet{size=0}} 01:51:02,928 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-1-shard-people-testWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4cf1fd02}, purged=MutableUnsignedLongSet{size=0}} 01:51:02,929 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 01:51:02,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 01:51:02,929 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:02,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:02,929 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2: commit - successfulFromPrevious: [] 01:51:02,929 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 01:51:02,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1: commit - successfulFromPrevious: [] 01:51:02,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:51:02,929 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-1663322862] with name member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher 01:51:02,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#1873455043] with name member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher 01:51:02,929 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:02,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:02,929 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-1663322862] 01:51:02,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:51:02,929 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:51:02,929 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:51:02,929 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#1873455043] 01:51:02,929 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:51:02,929 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher: 134.7 μs 01:51:02,929 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher: 92.55 μs 01:51:02,929 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-1663322862] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:02,930 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#1873455043] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:02,930 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#349046731]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2}} 01:51:02,930 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#349046731]}, enqueuedTicks=809253502584, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2}} 01:51:02,930 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} doCommit completed 01:51:02,930 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3068f389 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 01:51:02,930 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR}: initiating purge 01:51:02,930 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 3 01:51:02,930 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#349046731]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@15acbcac 01:51:02,930 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} 01:51:02,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Commit index set to 0 01:51:02,930 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED}: initiating purge 01:51:02,930 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} allocated sequence 3 01:51:02,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:51:02,930 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#349046731]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@24f0bbd7 01:51:02,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 01:51:02,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:51:02,930 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1398213343], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:02,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=180} 01:51:02,930 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d77bd70 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#349046731]}, enqueuedTicks=809265601009} 01:51:02,930 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#349046731]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2}} 01:51:02,930 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#349046731]}, enqueuedTicks=809253404013, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2}} 01:51:02,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:02,931 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 01:51:02,931 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6cb48f2d completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 01:51:02,931 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 01:51:02,931 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 01:51:02,931 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#349046731]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@2cd35abd 01:51:02,931 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} 01:51:02,931 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED}: initiating purge 01:51:02,931 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} allocated sequence 3 01:51:02,931 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#349046731]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801302238@5c6aca57 01:51:02,931 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#718980146], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:02,931 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@48c1af5 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#349046731]}, enqueuedTicks=809265816173} 01:51:02,931 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Commit index set to 0 01:51:02,931 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:51:02,931 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 01:51:02,931 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:51:02,931 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=176} 01:51:02,931 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:02,931 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@516a69de: creating a new snapshot member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:51:02,931 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1eee96cf}, purged=MutableUnsignedLongSet{size=0}} 01:51:02,931 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 01:51:02,931 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:02,931 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-1917029329] with name member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher 01:51:02,931 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:51:02,931 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:51:02,931 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-1917029329] 01:51:02,931 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:51:02,931 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@2e5d80a1 01:51:02,931 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:51:02,931 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@48c1af5 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=809266438243} 01:51:02,931 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher: 58.02 μs 01:51:02,931 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-1917029329] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:02,931 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-people-testWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@599758df}, purged=MutableUnsignedLongSet{size=0}} 01:51:02,931 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 01:51:02,931 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:02,932 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#909529099] with name member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher 01:51:02,932 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:51:02,932 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#909529099] 01:51:02,932 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:51:02,932 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#909529099] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:02,932 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher: 80.15 μs 01:51:02,933 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: purging transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:02,933 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 01:51:02,933 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 01:51:02,933 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:02,933 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: purging transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:02,933 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 01:51:02,933 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 01:51:02,933 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:02,933 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,933 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}]]] 01:51:02,933 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithMultipleShards: allocating new open snapshot member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1 01:51:02,934 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}]]] 01:51:02,934 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:51:02,934 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:51:02,934 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68} 01:51:02,934 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 01:51:02,934 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Log size is now 2 01:51:02,934 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:02,934 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:51:02,934 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:51:02,936 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithMultipleShards_slicer-id-725: slice: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 01:51:02,936 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 01:51:02,936 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 01:51:02,936 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:51:02,936 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 01:51:02,936 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:51:02,936 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68} 01:51:02,936 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: replication of member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@259f2a51 01:51:02,936 PM [cluster-test-shard-dispatcher-25] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithMultipleShards: finished purging transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:02,936 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SlicedMessageState - member-1-shard-cars-testWriteTransactionWithMultipleShards_slicer-id-725: Message size: 332 bytes, total slices to send: 1 01:51:02,936 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithMultipleShards_slicer-id-725: Message does not need to be sliced - sending original message 01:51:02,936 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 01:51:02,936 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:02,936 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:51:02,936 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:51:02,939 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#349046731]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} 01:51:02,939 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#349046731]}, enqueuedTicks=809265816173, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} 01:51:02,939 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE}: purge completed 01:51:02,939 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7bd4060 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} 01:51:02,939 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:51:02,939 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6cb48f2d purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 01:51:02,939 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 01:51:02,939 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=809266438243, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 01:51:02,939 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0} 01:51:02,939 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 01:51:02,939 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:51:02,939 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} allocated sequence 0 01:51:02,939 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d4a28@24fcde49 01:51:02,939 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d77bd70 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people}, enqueuedTicks=809274357060} 01:51:02,940 PM [cluster-test-shard-dispatcher-25] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testWriteTransactionWithMultipleShards: allocating new open snapshot member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2 01:51:02,940 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-people-testWriteTransactionWithMultipleShards_slicer-id-726: slice: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, message: Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 01:51:02,940 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SlicedMessageState - member-1-shard-people-testWriteTransactionWithMultipleShards_slicer-id-726: Message size: 337 bytes, total slices to send: 1 01:51:02,940 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-people-testWriteTransactionWithMultipleShards_slicer-id-726: Message does not need to be sliced - sending original message 01:51:02,940 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people} with Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 01:51:02,940 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#349046731], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people}, enqueuedTicks=809274357060, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 01:51:02,940 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0} 01:51:02,940 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:51:02,941 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,941 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:02,941 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onShutDown 01:51:02,941 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithMultipleShards - waiting on 2 shards 01:51:02,941 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Initiating leader transfer 01:51:02,941 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7173ede7 01:51:02,941 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onShutDown 01:51:02,941 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Initiating leader transfer 01:51:02,941 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7173ede7 01:51:02,941 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 01:51:02,941 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:02,941 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:51:02,941 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@ee6ba1b 01:51:02,941 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:02,941 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:02,941 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:51:02,941 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:02,941 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:02,941 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@ee6ba1b 01:51:02,941 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 01:51:02,941 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:51:02,941 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:02,941 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}]]] 01:51:02,942 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards] 01:51:02,942 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Commit index set to 1 01:51:02,942 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:51:02,942 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 01:51:02,942 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:51:02,942 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68} 01:51:02,942 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}]]] 01:51:02,942 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:51:02,942 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 01:51:02,942 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:51:02,942 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68} 01:51:02,942 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 01:51:02,942 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:02,942 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:51:02,942 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Log size is now 2 01:51:02,942 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:51:02,942 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:02,942 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:51:02,942 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 01:51:02,942 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:02,942 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards] 01:51:02,942 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:02,943 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@48c1af5 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@6f18aedc, splicing queue 01:51:02,943 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@48c1af5 queue spliced 0 messages 01:51:02,943 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#718980146], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#718980146], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:02,943 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: refreshing backend for shard 1 01:51:02,943 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#718980146], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:02,943 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:02,943 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:02,943 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-894855234] 01:51:02,943 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:51:02,943 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:51:02,943 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:02,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:51:02,943 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Leader transfer complete 01:51:02,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:51:02,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 01:51:02,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:51:02,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68} 01:51:02,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: replication of member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2224/0x00000008016adcd0@17af259 01:51:02,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testWriteTransactionWithMultipleShards: finished purging transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:02,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 01:51:02,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:02,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:51:02,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:02,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:51:02,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:51:02,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:02,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Leader transfer complete 01:51:02,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:51:02,943 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:02,943 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:51:02,944 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:51:02,944 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:02,944 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 01:51:02,944 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d77bd70 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@64026191, splicing queue 01:51:02,944 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1398213343], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:51:02,944 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5d77bd70 queue spliced 1 messages 01:51:02,944 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1398213343], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1398213343], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:02,944 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: refreshing backend for shard 2 01:51:02,944 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1398213343], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:51:02,944 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:51:02,944 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:51:02,944 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-894855234] 01:51:02,944 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:51:02,944 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:51:02,944 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} not found in inflight queue, checking pending queue 01:51:02,944 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Commit index set to 1 01:51:02,944 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#349046731]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} 01:51:02,944 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:51:02,944 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#349046731]}, enqueuedTicks=809265601009, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} 01:51:02,944 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 01:51:02,944 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE}: purge completed 01:51:02,944 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:51:02,944 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@57711114 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} 01:51:02,944 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68} 01:51:02,944 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR}: purge completed 01:51:02,944 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3068f389 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 01:51:02,944 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 01:51:02,944 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:02,944 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:51:02,945 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:51:03,358 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 01:51:03,358 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#718980146]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:51:03,358 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:03,358 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader)_slicer-id-731: Closing 01:51:03,358 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:03,358 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader)_slicer-id-730: Closing 01:51:03,358 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:51:03,359 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:51:03,359 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithMultipleShards 01:51:03,359 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 01:51:03,359 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:51:03,359 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 01:51:03,365 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 01:51:03,366 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:03,366 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:03,370 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:03,371 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:03,371 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:03,371 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:03,373 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:51:03,378 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 01:51:03,379 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:03,379 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Closing 01:51:03,379 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:03,379 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Closing 01:51:03,379 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:51:03,379 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:51:03,379 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithMultipleShards 01:51:03,380 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:03,380 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:04,380 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-7269894993774517783] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:04,395 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:04,396 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:04,398 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:04.399] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:04,400 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [8839479926786995824] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:04,406 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:04,407 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:04,409 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:04.410] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config